Merge remote-tracking branch 'upstream/debian/sid' into debian/sid

.travis.yml

@@ -2,5 +2,4 @@ language: cpp
 before_install:
  - sudo apt-get update -q
  - sudo ./prepare-release travis-ci
- - sudo apt-get install -q --no-install-recommends stunnel4 db-util
 script: make && make test && test/integration/run-tests

COMPILING

@@ -53,7 +53,7 @@ Debian GNU Linux 'potato'
 Debian GNU Linux 'woody'
   * All Archs
   - Works flawlessly
-  - You will want to have debiandoc-sgml and docbook2man installed to get
+  - You will want to have docbook-xml and docbook2man installed to get
     best results.
   - No IPv6 Support in glibc's < 2.1.
 

Makefile

@@ -10,7 +10,7 @@ endif
 default: startup all
 
 .PHONY: headers library clean veryclean all binary program doc test update-po
-all headers library clean veryclean binary program doc manpages debiandoc test update-po startup dirs:
+all headers library clean veryclean binary program doc manpages docbook test update-po startup dirs:
 	$(MAKE) -C vendor $@
 	$(MAKE) -C apt-pkg $@
 	$(MAKE) -C apt-inst $@
@@ -23,7 +23,7 @@ all headers library clean veryclean binary program doc manpages debiandoc test u
 	$(MAKE) -C po $@
 	$(MAKE) -C test $@
 
-all headers library clean veryclean binary program doc manpages debiandoc test update-po: startup dirs
+all headers library clean veryclean binary program doc manpages docbook test update-po: startup dirs
 
 dirs: startup
 

README.ddtp

@@ -1,74 +0,0 @@
-TODO:
-- URL-Remap for the translation files (to hack around the problem that
-  they are not on any ftp server yet but only on http://ddtp.debian.org/)
-
-Here is the original announcement of the ddtp support:
-
-* To: debian-devel-announce@lists.debian.org
-* Subject: Translate files
-* From: Michael Bramer <grisu@debian.org>
-* Date: Sun, 6 Oct 2002 21:56:06 +0200
-* Mail-followup-to: debian-devel@lists.debian.org
-* Message-id: <20021006195605.GA30516@home.debsupport.de>
-* Old-return-path: <michael@home.debsupport.de>
-* User-agent: Mutt/1.3.28i
-
-Hello all
-
-After some discussion between Anthony Towns (a ftpmaster), Jason
-Gunthorpe (APT Developer) and some DDTP Coordinators we find a way to
-transfer the translated package descriptions from the archive to the
-user. 
-
-The translated descriptions need to be downloadable befor any
-installation process, like the other package meta information. We
-choose a new file per languages with all translated package
-descriptions. The package system can download one or more of this
-files at 'apt-get update' time and know the translations.
-
-The new files are names 'Translate-$lang' and the file have this
-rfc822-format:
-  Package: &lt;package-name&gt;
-  Description-md5: &lt;the md5 checksum of the english description&gt;
-  Description-$lang.$encoding: &lt;translated headline&gt;
-   &lt;translated section&gt;
-
-The encoding of the Description is 'UTF-8' in all languages normal.
-The files will be located at 'debian/dists/sid/main/i18n/' on the ftp
-server (for all architecture). In addition of the plain
-'Translate-$lang' file, there will be a 'gz' and a 'bz2' version and
-in future also the new incremental format version.
-
-The &lt;the md5 checksum of the english description&gt; is the md5 checksum
-of the full english description, without the 'Description: '-tag and
-with all spaces and newlines. Look at this example:
- Description: XXX
-  YYY
-  .
-  ZZZ
-is md5(&quot;XXX\n YYY\n .\n ZZZ\n&quot;) (perl-syntax).
-
-
-A future APT version will download one or some 'Translate-$lang'
-file(s) at 'update'-time. After this download it show a translated
-description instead of the english form, if it found a translated
-description of the package with the right md5 chechsum. The environment
-of the user will controlled this process (LANG, LANGUAGE, LC_MESSAGES,
-etc). With this the package system will never show a outdated
-translation.
-
-The translations come all from the DDTP. A daily process on
-ddtp.debian.org make new 'Translated-$lang' files and a script on
-ftp-master request this files and move this to the debian archive.
-Now the first files are accessible at
-	<a  href="http://ddtp.debian.org/pdesc/translatefiles/">http://ddtp.debian.org/pdesc/translatefiles/</a>
-
-If you found wrong translations, please read the guides on
-ddtp.debian.org, make a better translation and send this per mail to
-the DDTP server. Don't bug the package maintainer!
-
-Thanks
-Grisu
-Michael Bramer  -  a Debian Linux Developer      <a  href="http://www.debsupport.de">http://www.debsupport.de</a>
-PGP: finger grisu@db.debian.org  -- Linux Sysadmin   -- Use Debian Linux

README.md

@@ -0,0 +1,181 @@
+APT
+===
+
+apt is the main commandline package manager for Debian and its derivatives.
+It provides commandline tools for searching and managing as well as querying
+information about packages as well as low-level access to all features
+provided by the libapt-pkg and libapt-inst libraries which higher-level
+package managers can depend upon.
+
+Included tools are:
+
+* apt-get for retrieval of packages and information about them
+  from authenticated sources and for installation, upgrade and
+  removal of packages together with their dependencies
+* apt-cache for querying available information about installed
+  as well as installable packages
+* apt-cdrom to use removable media as a source for packages
+* apt-config as an interface to the configuration settings
+* apt-key as an interface to manage authentication keys
+* apt-extracttemplates to be used by debconf to prompt for configuration
+  questions before installation.
+* apt-ftparchive creates Packages and other index files
+  needed to publish an archive of debian packages
+* apt-sortpkgs is a Packages/Sources file normalizer.
+
+The libraries libapt-pkg and libapt-inst are also maintained as part of this project,
+alongside various additional binaries like the acquire-methods used by them.
+Bindings for Python ([python-apt](https://tracker.debian.org/pkg/python-apt)) and
+Perl ([libapt-pkg-perl](https://tracker.debian.org/pkg/libapt-pkg-perl)) are available as separated projects.
+
+Discussion happens mostly on [the mailinglist](mailto:deity@lists.debian.org) ([archive](https://lists.debian.org/deity/)) and on [IRC](irc://irc.oftc.net/debian-apt).
+Our bugtracker as well as a general overview can be found at the [Debian Tracker page](https://tracker.debian.org/pkg/apt).
+
+
+Contributing
+------------
+APT is maintained in git, the official repository being located at
+`git://anonscm.debian.org/apt/apt.git` ([webgit](http://anonscm.debian.org/gitweb/?p=apt/apt.git)),
+but also available at other locations like [GitHub](https://github.com/Debian/apt).
+
+The default branch is `debian/sid`, other branches targeted at different
+derivatives and releases being used as needed. Various topic branches in
+different stages of completion might be branched of from those, which you
+are encouraged to do as well.
+
+### Coding
+
+APT uses its own autoconf based build system, see [README.make](http://anonscm.debian.org/gitweb/?p=apt/apt.git;a=blob;f=README.make)
+for the glory details, but to get started, just run:
+
+	$ make
+
+from a fresh git checkout.
+
+The source code uses in most parts a relatively uncommon indent convention,
+namely 3 spaces with 8 space tab (see [doc/style.txt](http://anonscm.debian.org/gitweb/?p=apt/apt.git;a=blob;f=doc/style.txt) for more on this).
+Adhering to it avoids unnecessary code-churn destroying history (aka: `git blame`)
+and you are therefore encouraged to write patches in this style.
+Your editor can surely help you with this, for vim the settings would be
+`setlocal shiftwidth=3 noexpandtab tabstop=8`
+(the later two are the default configuration and could therefore be omitted).
+
+### Translations
+
+While we welcome contributions here, we highly encourage you to contact the [Debian Internationalization (i18n) team](https://wiki.debian.org/Teams/I18n).
+Various language teams have formed which can help you creating, maintaining
+and improving a translation, while we could only do a basic syntax check of the
+file format…
+
+Further more, Translating APT is split into two independent parts:
+The program translation, meaning the messages printed by the tools,
+as well as the manpages and other documentation shipped with APT.
+
+### Bug triage
+
+Software tools like APT which are used by thousands of users every
+day have a steady flow of incoming bugreports. Not all of them are really
+bugs in APT: It can be packaging bugs like failing maintainer scripts a
+user reports against apt, because apt was the command he executed leading
+to this failure or various wishlist items for new features. Given enough time
+also the occasional duplicate enters the system.
+Our bugtracker is therefore full with open bugreports which are waiting for you! ;)
+
+Testing
+-------
+
+### Manual execution
+
+When you make changes and want to run them manually, make sure your
+`$LD_LIBRARY_PATH` points to the libraries you have built, e.g. via:
+
+	$ export LD_LIBRARY_PATH=$(pwd)/build/bin
+	$ ./build/bin/apt-get moo
+
+
+### Integration tests
+
+There is a extensive integration testsuite available which can be run via:
+
+	$ ./test/integration/run-tests
+
+While these tests are not executed at package build-time as they require additional
+dependencies, the repository contains the configuration needed to run them on [Travis CI](https://travis-ci.org/)
+as well as via autopkgtests e.g. on [Debian Continuous Integration](http://ci.debian.net/?q=apt#package/apt).
+
+A testcase here is a shellscript embedded in a framework creating an environment in which
+apt tools can be used naturally without root-rights to test every aspect of its behavior
+itself as well as in conjunction with dpkg and other tools while working with packages.
+
+
+### Unit tests
+
+These tests are gtest-dev based, reside in `./test/libapt` and can be run with `make test`.
+They are executed at package build-time, but not by `make`.
+
+Debugging
+---------
+
+APT does many things, so there is no central debug mode which could be
+activated. It uses instead various config-options to activate debug output
+in certain areas. The following describes some common scenarios and generally
+useful options, but is in no way exhaustive.
+
+Note that you should *NEVER* use these settings as root to avoid accidents.
+Similation mode (`-s`) is usually sufficient to help you run apt as a non-root user.
+
+### Using different state files
+
+If a dependency solver bug is reported, but can't be reproduced by the
+triager easily, it is beneficial to ask the reporter for the
+`/var/lib/dpkg/status` file, which includes the packages installed on the
+system and in which version. Such a file can then be used via the option
+`dir::state::status`. Beware of different architecture settings!
+Bugreports usually include this information in the template. Assuming you
+already have the `Packages` files for the architecture (see `sources.list`
+manpage for the `arch=` option) you can change to a different architecture
+with a config file like:
+
+	APT::Architecture "arch1";
+	#clear APT::Architectures;
+	APT:: Architectures { "arch1"; "arch2"; }
+
+If a certain mirror state is needed, see if you can reproduce it with [snapshot.debian.org](http://snapshot.debian.org/).
+Your sources.list file (`dir::etc::sourcelist`) has to be correctly mention the repository,
+but if it does, you can use different downloaded archive state files via `dir::state::lists`.
+
+In case manually vs. automatically installed matters, you can ask the reporter for
+the `/var/lib/apt/extended_states` file and use it with `dir::state::extended_states`.
+
+### Dependency resolution
+
+APT works in its internal resolver in two stages: First all packages are visited
+and marked for installation, keep back or removal. Option `Debug::pkgDepCache::Marker`
+shows this. This also decides which packages are to be installed to satisfy dependencies,
+which can be seen by `Debug::pkgDepCache::AutoInstall`. After this is done, we might
+be in a situation in which two packages want to be installed, but only on of them can be.
+It is the job of the pkgProblemResolver to decide which of two packages 'wins' and can
+therefore decide what has to happen. You can see the contenders as well as their fight and
+the resulting resolution with `Debug::pkgProblemResolver`.
+
+### Downloading files
+
+Various binaries (called 'methods') are tasked with downloading files. The Acquire system
+talks to them via simple text protocol. Depending on which side you want to see, either
+`Debug::pkgAcquire::Worker` or `Debug::Acquire::http` (or similar) will show the messages.
+
+The integration tests use a simple self-built webserver which also logs. If you find that
+the http(s) methods do not behave like they should be try to implement this behavior in the
+webserver for simpler and more controlled testing.
+
+### Installation order
+
+Dependencies are solved, packages downloaded: Everything read for the installation!
+The last step in the chain is often forgotten, but still very important:
+Packages have to be installed in a particular order so that their dependencies are
+satisfied, but at the same time you don't want to install very important and optional
+packages at the same time if possible, so that a broken optional package does not
+block the correct installation of very important packages. Which option to use depends on
+if you are interested in the topology sorting (`Debug::pkgOrderList`), the dependency-aware
+cycle and unconfigured prevention (`Debug::pkgPackageManager`) or the actual calls
+to dpkg (`Debug::pkgDpkgPm`).

apt-pkg/acquire-item.cc

@@ -356,7 +356,7 @@ pkgAcqDiffIndex::pkgAcqDiffIndex(pkgAcquire *Owner,
    Desc.URI = URI + ".diff/Index";
 
    DestFile = _config->FindDir("Dir::State::lists") + "partial/";
-   DestFile += URItoFileName(URI) + string(".DiffIndex");
+   DestFile += URItoFileName(Desc.URI);
 
    if(Debug)
       std::clog << "pkgAcqDiffIndex: " << Desc.URI << std::endl;
@@ -392,7 +392,7 @@ pkgAcqDiffIndex::pkgAcqDiffIndex(pkgAcquire *Owner,
 string pkgAcqDiffIndex::Custom600Headers()
 {
    string Final = _config->FindDir("Dir::State::lists");
-   Final += URItoFileName(RealURI) + string(".IndexDiff");
+   Final += URItoFileName(Desc.URI);
    
    if(Debug)
       std::clog << "Custom600Header-IMS: " << Final << std::endl;
@@ -932,8 +932,6 @@ pkgAcqIndex::pkgAcqIndex(pkgAcquire *Owner,
    }
    CompressionExtension = comprExt;
 
-   Verify = true;
-
    Init(URI, URIDesc, ShortDesc);
 }
 pkgAcqIndex::pkgAcqIndex(pkgAcquire *Owner, IndexTarget const *Target,
@@ -957,13 +955,6 @@ pkgAcqIndex::pkgAcqIndex(pkgAcquire *Owner, IndexTarget const *Target,
    if (CompressionExtension.empty() == false)
       CompressionExtension.erase(CompressionExtension.end()-1);
 
-   // only verify non-optional targets, see acquire-item.h for a FIXME
-   // to make this more flexible
-   if (Target->IsOptional())
-     Verify = false;
-   else
-     Verify = true;
-
    Init(Target->URI, Target->Description, Target->ShortDesc);
 }
 									/*}}}*/
@@ -978,8 +969,10 @@ void pkgAcqIndex::Init(string const &URI, string const &URIDesc, string const &S
    std::string const comprExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
    if (comprExt == "uncompressed")
       Desc.URI = URI;
-   else
+   else {
       Desc.URI = URI + '.' + comprExt;
+      DestFile = DestFile + '.' + comprExt;
+   }
 
    Desc.Description = URIDesc;
    Desc.Owner = this;
@@ -993,10 +986,11 @@ void pkgAcqIndex::Init(string const &URI, string const &URIDesc, string const &S
 /* The only header we use is the last-modified header. */
 string pkgAcqIndex::Custom600Headers()
 {
+   std::string const compExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
    string Final = _config->FindDir("Dir::State::lists");
    Final += URItoFileName(RealURI);
    if (_config->FindB("Acquire::GzipIndexes",false))
-      Final += ".gz";
+      Final += compExt;
    
    string msg = "\nIndex-File: true";
    // FIXME: this really should use "IndexTarget::IsOptional()" but that
@@ -1030,6 +1024,31 @@ void pkgAcqIndex::Failed(string Message,pkgAcquire::MethodConfig *Cnf)	/*{{{*/
    Item::Failed(Message,Cnf);
 }
 									/*}}}*/
+// pkgAcqIndex::GetFinalFilename - Return the full final file path      /*{{{*/
+std::string pkgAcqIndex::GetFinalFilename(std::string const &URI,
+                                          std::string const &compExt)
+{
+   std::string FinalFile = _config->FindDir("Dir::State::lists");
+   FinalFile += URItoFileName(URI);
+   if (_config->FindB("Acquire::GzipIndexes",false) == true)
+      FinalFile += '.' + compExt;
+   return FinalFile;
+}
+									/*}}}*/
+// AcqIndex::ReverifyAfterIMS - Reverify index after an ims-hit		/*{{{*/
+void pkgAcqIndex::ReverifyAfterIMS(std::string const &FileName)
+{
+   std::string const compExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
+   if (_config->FindB("Acquire::GzipIndexes",false) == true)
+      DestFile += compExt;
+
+   string FinalFile = GetFinalFilename(RealURI, compExt);
+   Rename(FinalFile, FileName);
+   Decompression = true;
+   Desc.URI = "copy:" + FileName;
+   QueueURI(Desc);
+}
+									/*}}}*/
 // AcqIndex::Done - Finished a fetch					/*{{{*/
 // ---------------------------------------------------------------------
 /* This goes through a number of states.. On the initial fetch the
@@ -1041,6 +1060,7 @@ void pkgAcqIndex::Done(string Message,unsigned long long Size,string Hash,
 		       pkgAcquire::MethodConfig *Cfg)
 {
    Item::Done(Message,Size,Hash,Cfg);
+   std::string const compExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
 
    if (Decompression == true)
    {
@@ -1052,33 +1072,34 @@ void pkgAcqIndex::Done(string Message,unsigned long long Size,string Hash,
 
       if (!ExpectedHash.empty() && ExpectedHash.toStr() != Hash)
       {
+         Desc.URI = RealURI;
 	 RenameOnError(HashSumMismatch);
          return;
       }
 
-      /* Verify the index file for correctness (all indexes must
-       * have a Package field) (LP: #346386) (Closes: #627642) */
-      if (Verify == true)
+      // FIXME: this can go away once we only ever download stuff that
+      //        has a valid hash and we never do GET based probing
+      //
+      /* Always verify the index file for correctness (all indexes must
+       * have a Package field) (LP: #346386) (Closes: #627642) 
+       */
+      FileFd fd(DestFile, FileFd::ReadOnly, FileFd::Extension);
+      // Only test for correctness if the file is not empty (empty is ok)
+      if (fd.Size() > 0)
       {
-	 FileFd fd(DestFile, FileFd::ReadOnly);
-	 // Only test for correctness if the file is not empty (empty is ok)
-	 if (fd.FileSize() > 0)
-	 {
-	    pkgTagSection sec;
-	    pkgTagFile tag(&fd);
-
-	    // all our current indexes have a field 'Package' in each section
-	    if (_error->PendingError() == true || tag.Step(sec) == false || sec.Exists("Package") == false)
-	    {
-	       RenameOnError(InvalidFormat);
-	       return;
-	    }
+         pkgTagSection sec;
+         pkgTagFile tag(&fd);
+         
+         // all our current indexes have a field 'Package' in each section
+         if (_error->PendingError() == true || tag.Step(sec) == false || sec.Exists("Package") == false)
+         {
+            RenameOnError(InvalidFormat);
+            return;
          }
       }
        
       // Done, move it into position
-      string FinalFile = _config->FindDir("Dir::State::lists");
-      FinalFile += URItoFileName(RealURI);
+      string FinalFile = GetFinalFilename(RealURI, compExt);
       Rename(DestFile,FinalFile);
       chmod(FinalFile.c_str(),0644);
       
@@ -1086,7 +1107,9 @@ void pkgAcqIndex::Done(string Message,unsigned long long Size,string Hash,
          will work OK */
       DestFile = _config->FindDir("Dir::State::lists") + "partial/";
       DestFile += URItoFileName(RealURI);
-      
+      if (_config->FindB("Acquire::GzipIndexes",false))
+         DestFile += '.' + compExt;
+
       // Remove the compressed version.
       if (Erase == true)
 	 unlink(DestFile.c_str());
@@ -1100,9 +1123,6 @@ void pkgAcqIndex::Done(string Message,unsigned long long Size,string Hash,
    string FileName = LookupTag(Message,"Alt-Filename");
    if (FileName.empty() == false)
    {
-      // The files timestamp matches
-      if (StringToBool(LookupTag(Message,"Alt-IMS-Hit"),false) == true)
-	 return;
       Decompression = true;
       Local = true;
       DestFile += ".decomp";
@@ -1119,33 +1139,43 @@ void pkgAcqIndex::Done(string Message,unsigned long long Size,string Hash,
       ErrorText = "Method gave a blank filename";
    }
 
-   std::string const compExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
-
-   // The files timestamp matches
-   if (StringToBool(LookupTag(Message,"IMS-Hit"),false) == true) {
-       if (_config->FindB("Acquire::GzipIndexes",false) && compExt == "gz")
-	  // Update DestFile for .gz suffix so that the clean operation keeps it
-	  DestFile += ".gz";
-      return;
-    }
-
    if (FileName == DestFile)
       Erase = true;
    else
       Local = true;
-   
+
+   // The files timestamp matches, for non-local URLs reverify the local
+   // file, for local file, uncompress again to ensure the hashsum is still
+   // matching the Release file
+   bool const IsCDROM = RealURI.substr(0,6) == "cdrom:";
+   if ((Local == false || IsCDROM == true) &&
+	 StringToBool(LookupTag(Message,"IMS-Hit"),false) == true)
+   {
+      // set destfile to the final destfile
+      if(_config->FindB("Acquire::GzipIndexes",false) == false)
+      {
+         DestFile = _config->FindDir("Dir::State::lists") + "partial/";
+         DestFile += URItoFileName(RealURI);
+      }
+
+      // do not reverify cdrom sources as apt-cdrom may rewrite the Packages
+      // file when its doing the indexcopy
+      if (IsCDROM == false)
+	 ReverifyAfterIMS(FileName);
+      return;
+   }
    string decompProg;
 
-   // If we enable compressed indexes and already have gzip, keep it
-   if (_config->FindB("Acquire::GzipIndexes",false) && compExt == "gz" && !Local) {
-      string FinalFile = _config->FindDir("Dir::State::lists");
-      FinalFile += URItoFileName(RealURI) + ".gz";
-      Rename(DestFile,FinalFile);
-      chmod(FinalFile.c_str(),0644);
-      
-      // Update DestFile for .gz suffix so that the clean operation keeps it
-      DestFile = _config->FindDir("Dir::State::lists") + "partial/";
-      DestFile += URItoFileName(RealURI) + ".gz";
+   // If we enable compressed indexes, queue for hash verification
+   if (_config->FindB("Acquire::GzipIndexes",false))
+   {
+      DestFile = _config->FindDir("Dir::State::lists");
+      DestFile += URItoFileName(RealURI) + '.' + compExt;
+
+      Decompression = true;
+      Desc.URI = "copy:" + FileName;
+      QueueURI(Desc);
+
       return;
     }
 
@@ -1186,8 +1216,11 @@ pkgAcqIndexTrans::pkgAcqIndexTrans(pkgAcquire *Owner, IndexTarget const *Target,
 // ---------------------------------------------------------------------
 string pkgAcqIndexTrans::Custom600Headers()
 {
+   std::string const compExt = CompressionExtension.substr(0, CompressionExtension.find(' '));
    string Final = _config->FindDir("Dir::State::lists");
    Final += URItoFileName(RealURI);
+   if (_config->FindB("Acquire::GzipIndexes",false))
+      Final += compExt;
 
    struct stat Buf;
    if (stat(Final.c_str(),&Buf) != 0)
@@ -1518,6 +1551,32 @@ void pkgAcqMetaIndex::AuthDone(string Message)				/*{{{*/
       std::cerr << "Signature verification succeeded: "
                 << DestFile << std::endl;
 
+   // do not trust any previously unverified content that we may have
+   string LastGoodSigFile = _config->FindDir("Dir::State::lists").append("partial/").append(URItoFileName(RealURI));
+   if (DestFile != SigFile)
+      LastGoodSigFile.append(".gpg");
+   LastGoodSigFile.append(".reverify");
+   if(IMSHit == false && RealFileExists(LastGoodSigFile) == false)
+   {
+      for (vector <struct IndexTarget*>::const_iterator Target = IndexTargets->begin();
+           Target != IndexTargets->end();
+           ++Target)
+      {
+         // remove old indexes
+         std::string index = _config->FindDir("Dir::State::lists") +
+            URItoFileName((*Target)->URI);
+         unlink(index.c_str());
+         // and also old gzipindexes
+         std::vector<std::string> types = APT::Configuration::getCompressionTypes();
+         for (std::vector<std::string>::const_iterator t = types.begin(); t != types.end(); ++t)
+         {
+            index += '.' + (*t);
+            unlink(index.c_str());
+         }
+      }
+   }
+
+
    // Download further indexes with verification
    QueueIndexes(true);
 

apt-pkg/acquire-item.h

@@ -685,15 +685,8 @@ class pkgAcqIndex : public pkgAcquire::Item
     */
    bool Erase;
 
-   /** \brief Verify for correctness by checking if a "Package"
-    *         tag is found in the index. This can be set to
-    *         false for optional index targets
-    *       
-    */
-   // FIXME: instead of a bool it should use a verify string that will
-   //        then be used in the pkgAcqIndex::Done method to ensure that
-   //        the downloaded file contains the expected tag
-   bool Verify;
+   // Unused, used to be used to verify that "Packages: " header was there
+   bool __DELME_ON_NEXT_ABI_BREAK_Verify;
 
    /** \brief The download request that is currently being
     *   processed.
@@ -713,6 +706,14 @@ class pkgAcqIndex : public pkgAcquire::Item
     */
    std::string CompressionExtension;
 
+   /** \brief Get the full pathname of the final file for the given URI
+    */
+   std::string GetFinalFilename(std::string const &URI,
+                                std::string const &compExt);
+
+   /** \brief Schedule file for verification after a IMS hit */
+   void ReverifyAfterIMS(std::string const &FileName);
+
    public:
    
    // Specialized action members

apt-pkg/acquire.cc

@@ -486,6 +486,9 @@ bool pkgAcquire::Clean(string Dir)
    if (DirectoryExists(Dir) == false)
       return true;
 
+   if(Dir == "/")
+      return _error->Error(_("Clean of %s is not supported"), Dir.c_str());
+
    DIR *D = opendir(Dir.c_str());   
    if (D == 0)
       return _error->Errno("opendir",_("Unable to read %s"),Dir.c_str());

apt-pkg/cachefile.cc

@@ -143,6 +143,9 @@ bool pkgCacheFile::BuildDepCache(OpProgress *Progress)
    if (DCache != NULL)
       return true;
 
+   if (BuildPolicy(Progress) == false)
+      return false;
+
    DCache = new pkgDepCache(Cache,Policy);
    if (_error->PendingError() == true)
       return false;

apt-pkg/cachefilter.h

@@ -73,6 +73,7 @@ public:
 	bool operator() (pkgCache::VerIterator const &Ver);
 	~PackageArchitectureMatchesSpecification();
 };
+									/*}}}*/
 
 #else
 

apt-pkg/clean.cc

@@ -34,7 +34,10 @@
 bool pkgArchiveCleaner::Go(std::string Dir,pkgCache &Cache)
 {
    bool CleanInstalled = _config->FindB("APT::Clean-Installed",true);
-      
+
+   if(Dir == "/")
+      return _error->Error(_("Clean of %s is not supported"), Dir.c_str());
+
    DIR *D = opendir(Dir.c_str());
    if (D == 0)
       return _error->Errno("opendir",_("Unable to read %s"),Dir.c_str());

apt-pkg/contrib/fileutl.h

@@ -85,7 +85,9 @@ class FileFd
    bool Skip(unsigned long long To);
    bool Truncate(unsigned long long To);
    unsigned long long Tell();
+   // the size of the file content (compressed files will be uncompressed first)
    unsigned long long Size();
+   // the size of the file itself
    unsigned long long FileSize();
    time_t ModificationTime();
 

apt-pkg/contrib/proxy.cc

@@ -0,0 +1,86 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/* ######################################################################
+   
+   Proxy - Proxy releated functions
+   
+   ##################################################################### */
+									/*}}}*/
+// Include Files							/*{{{*/
+#include<apt-pkg/configuration.h>
+#include<apt-pkg/error.h>
+#include<apt-pkg/fileutl.h>
+#include<apt-pkg/strutl.h>
+
+#include<iostream>
+#include <unistd.h>
+
+#include "proxy.h"
+
+
+// AutoDetectProxy - auto detect proxy          			/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+bool AutoDetectProxy(URI &URL)
+{
+   // we support both http/https debug options
+   bool Debug = _config->FindB("Debug::Acquire::"+URL.Access,false);
+
+   // the user already explicitly set a proxy for this host
+   if(_config->Find("Acquire::"+URL.Access+"::proxy::"+URL.Host, "") != "")
+      return true;
+
+   // option is "Acquire::http::Proxy-Auto-Detect" but we allow the old
+   // name without the dash ("-")
+   std::string AutoDetectProxyCmd = _config->Find("Acquire::"+URL.Access+"::Proxy-Auto-Detect",
+                                      _config->Find("Acquire::"+URL.Access+"::ProxyAutoDetect"));
+
+   if (AutoDetectProxyCmd.empty())
+      return true;
+
+   if (Debug)
+      std::clog << "Using auto proxy detect command: " << AutoDetectProxyCmd << std::endl;
+
+   int Pipes[2] = {-1,-1};
+   if (pipe(Pipes) != 0)
+      return _error->Errno("pipe", "Failed to create Pipe");
+
+   pid_t Process = ExecFork();
+   if (Process == 0)
+   {
+      close(Pipes[0]);
+      dup2(Pipes[1],STDOUT_FILENO);
+      SetCloseExec(STDOUT_FILENO,false);
+
+      std::string foo = URL;
+      const char *Args[4];
+      Args[0] = AutoDetectProxyCmd.c_str();
+      Args[1] = foo.c_str();
+      Args[2] = 0;
+      execv(Args[0],(char **)Args);
+      std::cerr << "Failed to exec method " << Args[0] << std::endl;
+      _exit(100);
+   }
+   char buf[512];
+   int InFd = Pipes[0];
+   close(Pipes[1]);
+   int res = read(InFd, buf, sizeof(buf)-1);
+   ExecWait(Process, "ProxyAutoDetect", true);
+
+   if (res < 0)
+      return _error->Errno("read", "Failed to read");
+   if (res == 0)
+      return _error->Warning("ProxyAutoDetect returned no data");
+
+   // add trailing \0
+   buf[res] = 0;
+
+   if (Debug)
+      std::clog << "auto detect command returned: '" << buf << "'" << std::endl;
+
+   if (strstr(buf, URL.Access.c_str()) == buf)
+      _config->Set("Acquire::"+URL.Access+"::proxy::"+URL.Host, _strstrip(buf));
+
+   return true;
+}
+									/*}}}*/

apt-pkg/contrib/proxy.h

@@ -0,0 +1,16 @@
+// -*- mode: cpp; mode: fold -*-
+// Description								/*{{{*/
+/* ######################################################################
+   
+   Proxy - Proxy operations
+   
+   ##################################################################### */
+									/*}}}*/
+#ifndef PKGLIB_PROXY_H
+#define PKGLIB_PROXY_H
+
+class URI;
+bool AutoDetectProxy(URI &URL);
+
+
+#endif

apt-pkg/contrib/strutl.cc

@@ -45,14 +45,26 @@ using namespace std;
 // ---------------------------------------------------------------------
 namespace APT {
    namespace String {
-std::string Strip(const std::string &s)
+std::string Strip(const std::string &str)
 {
-   size_t start = s.find_first_not_of(" \t\n");
-   // only whitespace
-   if (start == string::npos)
+   // ensure we have at least one character
+   if (str.empty() == true)
+      return str;
+
+   char const * const s = str.c_str();
+   size_t start = 0;
+   for (; isspace(s[start]) != 0; ++start)
+      ; // find the first not-space
+
+   // string contains only whitespaces
+   if (s[start] == '\0')
       return "";
-   size_t end = s.find_last_not_of(" \t\n");
-   return s.substr(start, end-start+1);
+
+   size_t end = str.length() - 1;
+   for (; isspace(s[end]) != 0; --end)
+      ; // find the last not-space
+
+   return str.substr(start, end - start + 1);
 }
 
 bool Endswith(const std::string &s, const std::string &end)
@@ -704,9 +716,12 @@ string LookupTag(const string &Message,const char *Tag,const char *Default)
    then returns the result. Several varients on true/false are checked. */
 int StringToBool(const string &Text,int Default)
 {
-   char *End;
-   int Res = strtol(Text.c_str(),&End,0);   
-   if (End != Text.c_str() && Res >= 0 && Res <= 1)
+   char *ParseEnd;
+   int Res = strtol(Text.c_str(),&ParseEnd,0);
+   // ensure that the entire string was converted by strtol to avoid
+   // failures on "apt-cache show -a 0ad" where the "0" is converted
+   const char *TextEnd = Text.c_str()+Text.size();
+   if (ParseEnd == TextEnd && Res >= 0 && Res <= 1)
       return Res;
    
    // Check for positives

apt-pkg/deb/debindexfile.cc

@@ -80,14 +80,18 @@ pkgSrcRecords::Parser *debSourcesIndex::CreateSrcParser() const
 {
    string SourcesURI = _config->FindDir("Dir::State::lists") + 
       URItoFileName(IndexURI("Sources"));
-   string SourcesURIgzip = SourcesURI + ".gz";
 
-   if (!FileExists(SourcesURI) && !FileExists(SourcesURIgzip))
-      return NULL;
-   else if (!FileExists(SourcesURI) && FileExists(SourcesURIgzip))
-      SourcesURI = SourcesURIgzip;
-
-   return new debSrcRecordParser(SourcesURI,this);
+   std::vector<std::string> types = APT::Configuration::getCompressionTypes();
+   for (std::vector<std::string>::const_iterator t = types.begin(); t != types.end(); ++t)
+   {
+      string p;
+      p = SourcesURI + '.' + *t;
+      if (FileExists(p))
+         return new debSrcRecordParser(p, this);
+   }
+   if (FileExists(SourcesURI))
+      return new debSrcRecordParser(SourcesURI, this);
+   return NULL;
 }
 									/*}}}*/
 // SourcesIndex::Describe - Give a descriptive path to the index	/*{{{*/
@@ -129,11 +133,15 @@ string debSourcesIndex::Info(const char *Type) const
 inline string debSourcesIndex::IndexFile(const char *Type) const
 {
    string s = URItoFileName(IndexURI(Type));
-   string sgzip = s + ".gz";
-   if (!FileExists(s) && FileExists(sgzip))
-       return sgzip;
-   else
-       return s;
+
+   std::vector<std::string> types = APT::Configuration::getCompressionTypes();
+   for (std::vector<std::string>::const_iterator t = types.begin(); t != types.end(); ++t)
+   {
+      string p = s + '.' + *t;
+      if (FileExists(p))
+         return p;
+   }
+   return s;
 }
 
 string debSourcesIndex::IndexURI(const char *Type) const
@@ -259,11 +267,15 @@ string debPackagesIndex::Info(const char *Type) const
 inline string debPackagesIndex::IndexFile(const char *Type) const
 {
    string s =_config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
-   string sgzip = s + ".gz";
-   if (!FileExists(s) && FileExists(sgzip))
-       return sgzip;
-   else
-       return s;
+
+   std::vector<std::string> types = APT::Configuration::getCompressionTypes();
+   for (std::vector<std::string>::const_iterator t = types.begin(); t != types.end(); ++t)
+   {
+      string p = s + '.' + *t;
+      if (FileExists(p))
+         return p;
+   }
+   return s;
 }
 string debPackagesIndex::IndexURI(const char *Type) const
 {
@@ -411,11 +423,15 @@ debTranslationsIndex::debTranslationsIndex(string URI,string Dist,string Section
 inline string debTranslationsIndex::IndexFile(const char *Type) const
 {
    string s =_config->FindDir("Dir::State::lists") + URItoFileName(IndexURI(Type));
-   string sgzip = s + ".gz";
-   if (!FileExists(s) && FileExists(sgzip))
-       return sgzip;
-   else
-       return s;
+
+   std::vector<std::string> types = APT::Configuration::getCompressionTypes();
+   for (std::vector<std::string>::const_iterator t = types.begin(); t != types.end(); ++t)
+   {
+      string p = s + '.' + *t;
+      if (FileExists(p))
+         return p;
+   }
+   return s;
 }
 string debTranslationsIndex::IndexURI(const char *Type) const
 {

apt-pkg/deb/deblistparser.cc

@@ -108,7 +108,7 @@ unsigned char debListParser::ParseMultiArch(bool const showErrors)	/*{{{*/
 {
    unsigned char MA;
    string const MultiArch = Section.FindS("Multi-Arch");
-   if (MultiArch.empty() == true)
+   if (MultiArch.empty() == true || MultiArch == "no")
       MA = pkgCache::Version::None;
    else if (MultiArch == "same") {
       if (ArchitectureAll() == true)
@@ -145,7 +145,8 @@ unsigned char debListParser::ParseMultiArch(bool const showErrors)	/*{{{*/
 bool debListParser::NewVersion(pkgCache::VerIterator &Ver)
 {
    // Parse the section
-   Ver->Section = UniqFindTagWrite("Section");
+   unsigned long const idxSection = UniqFindTagWrite("Section");
+   Ver->Section = idxSection;
    Ver->MultiArch = ParseMultiArch(true);
    // Archive Size
    Ver->Size = Section.FindULL("Size");
@@ -260,7 +261,10 @@ bool debListParser::UsePackage(pkgCache::PkgIterator &Pkg,
 			       pkgCache::VerIterator &Ver)
 {
    if (Pkg->Section == 0)
-      Pkg->Section = UniqFindTagWrite("Section");
+   {
+      unsigned long const idxSection = UniqFindTagWrite("Section");
+      Pkg->Section = idxSection;
+   }
 
    string const static myArch = _config->Find("APT::Architecture");
    // Possible values are: "all", "native", "installed" and "none"
@@ -334,13 +338,9 @@ unsigned short debListParser::VersionHash()
 /* Status lines are of the form,
      Status: want flag status
    want = unknown, install, hold, deinstall, purge
-   flag = ok, reinstreq, hold, hold-reinstreq
-   status = not-installed, unpacked, half-configured,
-            half-installed, config-files, post-inst-failed, 
-            removal-failed, installed
-   
-   Some of the above are obsolete (I think?) flag = hold-* and 
-   status = post-inst-failed, removal-failed at least.
+   flag = ok, reinstreq
+   status = not-installed, config-files, half-installed, unpacked,
+            half-configured, triggers-awaited, triggers-pending, installed
  */
 bool debListParser::ParseStatus(pkgCache::PkgIterator &Pkg,
 				pkgCache::VerIterator &Ver)
@@ -397,15 +397,13 @@ bool debListParser::ParseStatus(pkgCache::PkgIterator &Pkg,
 
    // Process the flag field
    WordList StatusList[] = {{"not-installed",pkgCache::State::NotInstalled},
+                            {"config-files",pkgCache::State::ConfigFiles},
+                            {"half-installed",pkgCache::State::HalfInstalled},
                             {"unpacked",pkgCache::State::UnPacked},
                             {"half-configured",pkgCache::State::HalfConfigured},
-                            {"installed",pkgCache::State::Installed},
-                            {"half-installed",pkgCache::State::HalfInstalled},
-                            {"config-files",pkgCache::State::ConfigFiles},
                             {"triggers-awaited",pkgCache::State::TriggersAwaited},
                             {"triggers-pending",pkgCache::State::TriggersPending},
-                            {"post-inst-failed",pkgCache::State::HalfConfigured},
-                            {"removal-failed",pkgCache::State::HalfInstalled},
+                            {"installed",pkgCache::State::Installed},
                             {NULL, 0}};
    if (GrabWord(string(Start,I-Start),StatusList,Pkg->CurrentState) == false)
       return _error->Error("Malformed 3rd word in the Status line");
@@ -631,72 +629,94 @@ const char *debListParser::ParseDepends(const char *Start,const char *Stop,
 
    if (ParseRestrictionsList == true)
    {
-      // Parse a restrictions list
-      if (I != Stop && *I == '<')
+      // Parse a restrictions formula which is in disjunctive normal form:
+      // (foo AND bar) OR (blub AND bla)
+
+      std::vector<string> const profiles = APT::Configuration::getBuildProfiles();
+
+      // if the next character is a restriction list, then by default the
+      // dependency does not apply and the conditions have to be checked
+      // if the next character is not a restriction list, then by default the
+      // dependency applies
+      bool applies1 = (*I != '<');
+      while (I != Stop)
       {
+	 if (*I != '<')
+	     break;
+
 	 ++I;
 	 // malformed
 	 if (unlikely(I == Stop))
 	    return 0;
 
-	 std::vector<string> const profiles = APT::Configuration::getBuildProfiles();
-
 	 const char *End = I;
-	 bool Found = false;
-	 bool NegRestriction = false;
-	 while (I != Stop)
-	 {
-	    // look for whitespace or ending '>'
-	    for (;End != Stop && !isspace(*End) && *End != '>'; ++End);
 
-	    if (unlikely(End == Stop))
-	       return 0;
-
-	    if (*I == '!')
+	 // if of the prior restriction list is already fulfilled, then
+	 // we can just skip to the end of the current list
+	 if (applies1) {
+	    for (;End != Stop && *End != '>'; ++End);
+	    I = ++End;
+	    // skip whitespace
+	    for (;I != Stop && isspace(*I) != 0; I++);
+	 } else {
+	    bool applies2 = true;
+	    // all the conditions inside a restriction list have to be
+	    // met so once we find one that is not met, we can skip to
+	    // the end of this list
+	    while (I != Stop)
 	    {
-	       NegRestriction = true;
-	       ++I;
-	    }
+	       // look for whitespace or ending '>'
+	       // End now points to the character after the current term
+	       for (;End != Stop && !isspace(*End) && *End != '>'; ++End);
 
-	    std::string restriction(I, End);
+	       if (unlikely(End == Stop))
+		  return 0;
 
-	    std::string prefix = "profile.";
-	    // only support for "profile" prefix, ignore others
-	    if (restriction.size() > prefix.size() &&
-		  restriction.substr(0, prefix.size()) == prefix)
-	    {
-	       // get the name of the profile
-	       restriction = restriction.substr(prefix.size());
+	       bool NegRestriction = false;
+	       if (*I == '!')
+	       {
+		  NegRestriction = true;
+		  ++I;
+	       }
+
+	       std::string restriction(I, End);
 
 	       if (restriction.empty() == false && profiles.empty() == false &&
-		     std::find(profiles.begin(), profiles.end(), restriction) != profiles.end())
+		  std::find(profiles.begin(), profiles.end(), restriction) != profiles.end())
 	       {
-		  Found = true;
-		  if (I[-1] != '!')
-		     NegRestriction = false;
-		  // we found a match, so fast-forward to the end of the wildcards
-		  for (; End != Stop && *End != '>'; ++End);
+		  if (NegRestriction) {
+		     applies2 = false;
+		     // since one of the terms does not apply we don't have to check the others
+		     for (; End != Stop && *End != '>'; ++End);
+		  }
+	       } else {
+		  if (!NegRestriction) {
+		     applies2 = false;
+		     // since one of the terms does not apply we don't have to check the others
+		     for (; End != Stop && *End != '>'; ++End);
+		  }
+	       }
+
+	       if (*End++ == '>') {
+		  I = End;
+		  // skip whitespace
+		  for (;I != Stop && isspace(*I) != 0; I++);
+		  break;
 	       }
-	    }
 
-	    if (*End++ == '>') {
 	       I = End;
-	       break;
+	       // skip whitespace
+	       for (;I != Stop && isspace(*I) != 0; I++);
+	    }
+	    if (applies2) {
+	       applies1 = true;
 	    }
-
-	    I = End;
-	    for (;I != Stop && isspace(*I) != 0; I++);
 	 }
-
-	 if (NegRestriction == true)
-	    Found = !Found;
-
-	 if (Found == false)
-	    Package = ""; /* not for this restriction */
       }
 
-      // Skip whitespace
-      for (;I != Stop && isspace(*I) != 0; I++);
+      if (applies1 == false) {
+	 Package = ""; //not for this restriction
+      }
    }
 
    if (I != Stop && *I == '|')
@@ -799,8 +819,8 @@ bool debListParser::ParseProvides(pkgCache::VerIterator &Ver)
 	 Start = ParseDepends(Start,Stop,Package,Version,Op);
 	 if (Start == 0)
 	    return _error->Error("Problem parsing Provides line");
-	 if (Op != pkgCache::Dep::NoOp) {
-	    _error->Warning("Ignoring Provides line with DepCompareOp for package %s", Package.c_str());
+	 if (Op != pkgCache::Dep::NoOp && Op != pkgCache::Dep::Equals) {
+	    _error->Warning("Ignoring Provides line with non-equal DepCompareOp for package %s", Package.c_str());
 	 } else if ((Ver->MultiArch & pkgCache::Version::Foreign) == pkgCache::Version::Foreign) {
 	    if (NewProvidesAllArch(Ver, Package, Version) == false)
 	       return false;

apt-pkg/deb/dpkgpm.cc

@@ -55,12 +55,24 @@
 
 using namespace std;
 
+APT_PURE static unsigned int
+EnvironmentSize()
+{
+  unsigned int size = 0;
+  char **envp = environ;
+
+  while (*envp != NULL)
+    size += strlen (*envp++) + 1;
+
+  return size;
+}
+
 class pkgDPkgPMPrivate 
 {
 public:
    pkgDPkgPMPrivate() : stdin_is_dev_null(false), dpkgbuf_pos(0),
-			term_out(NULL), history_out(NULL), 
-                        progress(NULL), master(-1), slave(-1)
+			term_out(NULL), history_out(NULL),
+                        progress(NULL), master(-1), slave(NULL)
    {
       dpkgbuf[0] = '\0';
    }
@@ -77,9 +89,9 @@ public:
    APT::Progress::PackageManager *progress;
 
    // pty stuff
-   struct	termios tt;
+   struct termios tt;
    int master;
-   int slave;
+   char * slave;
 
    // signals
    sigset_t sigmask;
@@ -510,7 +522,7 @@ bool pkgDPkgPM::RunScriptsWithPkgs(const char *Cnf)
    return result;
 }
 									/*}}}*/
-// DPkgPM::DoStdin - Read stdin and pass to slave pty			/*{{{*/
+// DPkgPM::DoStdin - Read stdin and pass to master pty			/*{{{*/
 // ---------------------------------------------------------------------
 /*
 */
@@ -564,8 +576,8 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
       'status:   <pkg>: <pkg  qstate>'
       'status:   <pkg>:<arch>: <pkg  qstate>'
       
-      'processing: {install,configure,remove,purge,disappear,trigproc}: pkg'
-      'processing: {install,configure,remove,purge,disappear,trigproc}: trigger'
+      'processing: {install,upgrade,configure,remove,purge,disappear,trigproc}: pkg'
+      'processing: {install,upgrade,configure,remove,purge,disappear,trigproc}: trigger'
    */
 
    // we need to split on ": " (note the appended space) as the ':' is
@@ -589,12 +601,15 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
    std::string action;
 
    // "processing" has the form "processing: action: pkg or trigger"
-   // with action = ["install", "configure", "remove", "purge", "disappear",
-   //                "trigproc"]
+   // with action = ["install", "upgrade", "configure", "remove", "purge",
+   //                "disappear", "trigproc"]
    if (prefix == "processing")
    {
       pkgname = APT::String::Strip(list[2]);
       action = APT::String::Strip(list[1]);
+      // we don't care for the difference (as dpkg doesn't really either)
+      if (action == "upgrade")
+	 action = "install";
    }
    // "status" has the form: "status: pkg: state"
    // with state in ["half-installed", "unpacked", "half-configured", 
@@ -621,15 +636,15 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
    {
       if(action == "error")
       {
-         d->progress->Error(list[1], PackagesDone, PackagesTotal,
+         d->progress->Error(pkgname, PackagesDone, PackagesTotal,
                             list[3]);
          pkgFailures++;
-         WriteApportReport(list[1].c_str(), list[3].c_str());
+         WriteApportReport(pkgname.c_str(), list[3].c_str());
          return;
       }
       else if(action == "conffile-prompt")
       {
-         d->progress->ConffilePrompt(list[1], PackagesDone, PackagesTotal,
+         d->progress->ConffilePrompt(pkgname, PackagesDone, PackagesTotal,
                                      list[3]);
          return;
       }
@@ -638,27 +653,26 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
    // at this point we know that we should have a valid pkgname, so build all 
    // the info from it
 
-   // dpkg does not send always send "pkgname:arch" so we add it here 
-   // if needed
+   // dpkg does not always send "pkgname:arch" so we add it here if needed
    if (pkgname.find(":") == std::string::npos)
    {
-      // find the package in the group that is in a touched by dpkg
-      // if there are multiple dpkg will send us a full pkgname:arch
+      // find the package in the group that is touched by dpkg
+      // if there are multiple pkgs dpkg would send us a full pkgname:arch
       pkgCache::GrpIterator Grp = Cache.FindGrp(pkgname);
-      if (Grp.end() == false) 
+      if (Grp.end() == false)
       {
-          pkgCache::PkgIterator P = Grp.PackageList();
-          for (; P.end() != true; P = Grp.NextPkg(P))
-          {
-              if(Cache[P].Mode != pkgDepCache::ModeKeep)
-              {
-                  pkgname = P.FullName();
-                  break;
-              }
-          }
+	 pkgCache::PkgIterator P = Grp.PackageList();
+	 for (; P.end() != true; P = Grp.NextPkg(P))
+	 {
+	    if(Cache[P].Keep() == false || Cache[P].ReInstall() == true)
+	    {
+	       pkgname = P.FullName();
+	       break;
+	    }
+	 }
       }
    }
-   
+
    const char* const pkg = pkgname.c_str();
    std::string short_pkgname = StringSplit(pkgname, ":")[0];
    std::string arch = "";
@@ -697,28 +711,29 @@ void pkgDPkgPM::ProcessDpkgStatusLine(char *line)
    if (prefix == "status")
    {
       vector<struct DpkgState> const &states = PackageOps[pkg];
-      const char *next_action = NULL;
       if(PackageOpsDone[pkg] < states.size())
-         next_action = states[PackageOpsDone[pkg]].state;
-      // check if the package moved to the next dpkg state
-      if(next_action && (action == next_action))
       {
-         // only read the translation if there is actually a next
-         // action
-         const char *translation = _(states[PackageOpsDone[pkg]].str);
-         std::string msg;
-
-         // we moved from one dpkg state to a new one, report that
-         PackageOpsDone[pkg]++;
-         PackagesDone++;
-
-         strprintf(msg, translation, i18n_pkgname.c_str());
-         d->progress->StatusChanged(pkgname, PackagesDone, PackagesTotal, msg);
-         
+         char const * const next_action = states[PackageOpsDone[pkg]].state;
+	 if (next_action && Debug == true)
+	    std::clog << "(parsed from dpkg) pkg: " << short_pkgname
+	       << " action: " << action << " (expected: '" << next_action << "' "
+	       << PackageOpsDone[pkg] << " of " << states.size() << ")" << endl;
+
+	 // check if the package moved to the next dpkg state
+	 if(next_action && (action == next_action))
+	 {
+	    // only read the translation if there is actually a next action
+	    char const * const translation = _(states[PackageOpsDone[pkg]].str);
+
+	    // we moved from one dpkg state to a new one, report that
+	    ++PackageOpsDone[pkg];
+	    ++PackagesDone;
+
+	    std::string msg;
+	    strprintf(msg, translation, i18n_pkgname.c_str());
+	    d->progress->StatusChanged(pkgname, PackagesDone, PackagesTotal, msg);
+	 }
       }
-      if (Debug == true) 
-         std::clog << "(parsed from dpkg) pkg: " << short_pkgname
-                   << " action: " << action << endl;
    }
 }
 									/*}}}*/
@@ -1048,60 +1063,127 @@ void pkgDPkgPM::StartPtyMagic()
 {
    if (_config->FindB("Dpkg::Use-Pty", true) == false)
    {
-      d->master = d->slave = -1;
+      d->master = -1;
+      if (d->slave != NULL)
+	 free(d->slave);
+      d->slave = NULL;
       return;
    }
 
-   // setup the pty and stuff
-   struct winsize win;
-
+   _error->PushToStack();
    // if tcgetattr for both stdin/stdout returns 0 (no error)
    // we do the pty magic
-   _error->PushToStack();
-   if (tcgetattr(STDIN_FILENO, &d->tt) == 0 &&
-       tcgetattr(STDOUT_FILENO, &d->tt) == 0)
+   if (tcgetattr(STDOUT_FILENO, &d->tt) == 0 &&
+	 tcgetattr(STDIN_FILENO, &d->tt) == 0)
    {
-       if (ioctl(STDOUT_FILENO, TIOCGWINSZ, (char *)&win) < 0)
-       {
-           _error->Errno("ioctl", _("ioctl(TIOCGWINSZ) failed"));
-       } else if (openpty(&d->master, &d->slave, NULL, &d->tt, &win) < 0)
-       {
-           _error->Errno("openpty", _("Can not write log (%s)"), _("Is /dev/pts mounted?"));
-           d->master = d->slave = -1;
-        } else {
-	    struct termios rtt;
-	    rtt = d->tt;
-	    cfmakeraw(&rtt);
-	    rtt.c_lflag &= ~ECHO;
-	    rtt.c_lflag |= ISIG;
+      d->master = posix_openpt(O_RDWR | O_NOCTTY);
+      if (d->master == -1)
+	 _error->Errno("posix_openpt", _("Can not write log (%s)"), _("Is /dev/pts mounted?"));
+      else if (unlockpt(d->master) == -1)
+      {
+	 _error->Errno("unlockpt", "Unlocking the slave of master fd %d failed!", d->master);
+	 close(d->master);
+	 d->master = -1;
+      }
+      else
+      {
+	 char const * const slave_name = ptsname(d->master);
+	 if (slave_name == NULL)
+	 {
+	    _error->Errno("unlockpt", "Getting name for slave of master fd %d failed!", d->master);
+	    close(d->master);
+	    d->master = -1;
+	 }
+	 else
+	 {
+	    d->slave = strdup(slave_name);
+	    if (d->slave == NULL)
+	    {
+	       _error->Errno("strdup", "Copying name %s for slave of master fd %d failed!", slave_name, d->master);
+	       close(d->master);
+	       d->master = -1;
+	    }
+	    struct winsize win;
+	    if (ioctl(STDOUT_FILENO, TIOCGWINSZ, &win) < 0)
+	       _error->Errno("ioctl", "Getting TIOCGWINSZ from stdout failed!");
+	    if (ioctl(d->master, TIOCSWINSZ, &win) < 0)
+	       _error->Errno("ioctl", "Setting TIOCSWINSZ for master fd %d failed!", d->master);
+	    if (tcsetattr(d->master, TCSANOW, &d->tt) == -1)
+	       _error->Errno("tcsetattr", "Setting in Start via TCSANOW for master fd %d failed!", d->master);
+
+	    struct termios raw_tt;
+	    raw_tt = d->tt;
+	    cfmakeraw(&raw_tt);
+	    raw_tt.c_lflag &= ~ECHO;
+	    raw_tt.c_lflag |= ISIG;
 	    // block SIGTTOU during tcsetattr to prevent a hang if
 	    // the process is a member of the background process group
 	    // http://www.opengroup.org/onlinepubs/000095399/functions/tcsetattr.html
 	    sigemptyset(&d->sigmask);
 	    sigaddset(&d->sigmask, SIGTTOU);
 	    sigprocmask(SIG_BLOCK,&d->sigmask, &d->original_sigmask);
-	    tcsetattr(0, TCSAFLUSH, &rtt);
-	    sigprocmask(SIG_SETMASK, &d->original_sigmask, 0);
-        }
+	    if (tcsetattr(STDIN_FILENO, TCSAFLUSH, &raw_tt) == -1)
+	       _error->Errno("tcsetattr", "Setting in Start via TCSAFLUSH for stdout failed!");
+	    sigprocmask(SIG_SETMASK, &d->original_sigmask, NULL);
+	 }
       }
-   // complain only if stdout is either a terminal (but still failed) or is an invalid
+   }
+   else
+   {
+      // complain only if stdout is either a terminal (but still failed) or is an invalid
       // descriptor otherwise we would complain about redirection to e.g. /dev/null as well.
-      else if (isatty(STDOUT_FILENO) == 1 || errno == EBADF)
-         _error->Errno("tcgetattr", _("Can not write log (%s)"), _("Is stdout a terminal?"));
+      if (isatty(STDOUT_FILENO) == 1 || errno == EBADF)
+	 _error->Errno("tcgetattr", _("Can not write log (%s)"), _("Is stdout a terminal?"));
+   }
 
-      if (_error->PendingError() == true)
-	 _error->DumpErrors(std::cerr);
-      _error->RevertToStack();
+   if (_error->PendingError() == true)
+   {
+      if (d->master != -1)
+      {
+	 close(d->master);
+	 d->master = -1;
+      }
+      _error->DumpErrors(std::cerr);
+   }
+   _error->RevertToStack();
 }
+void pkgDPkgPM::SetupSlavePtyMagic()
+{
+   if(d->master == -1)
+      return;
+
+   if (close(d->master) == -1)
+      _error->FatalE("close", "Closing master %d in child failed!", d->master);
+   if (setsid() == -1)
+      _error->FatalE("setsid", "Starting a new session for child failed!");
+
+   int const slaveFd = open(d->slave, O_RDWR);
+   if (slaveFd == -1)
+      _error->FatalE("open", _("Can not write log (%s)"), _("Is /dev/pts mounted?"));
+
+   if (ioctl(slaveFd, TIOCSCTTY, 0) < 0)
+      _error->FatalE("ioctl", "Setting TIOCSCTTY for slave fd %d failed!", slaveFd);
+   else
+   {
+      for (unsigned short i = 0; i < 3; ++i)
+	 if (dup2(slaveFd, i) == -1)
+	    _error->FatalE("dup2", "Dupping %d to %d in child failed!", slaveFd, i);
 
+      if (tcsetattr(0, TCSANOW, &d->tt) < 0)
+	 _error->FatalE("tcsetattr", "Setting in Setup via TCSANOW for slave fd %d failed!", slaveFd);
+   }
+}
 void pkgDPkgPM::StopPtyMagic()
 {
-   if(d->slave > 0)
-      close(d->slave);
+   if (d->slave != NULL)
+      free(d->slave);
+   d->slave = NULL;
    if(d->master >= 0) 
    {
-      tcsetattr(0, TCSAFLUSH, &d->tt);
+      if (tcsetattr(0, TCSAFLUSH, &d->tt) == -1)
+	 _error->FatalE("tcsetattr", "Setting in Stop via TCSAFLUSH for stdin failed!");
       close(d->master);
+      d->master = -1;
    }
 }
 
@@ -1166,8 +1248,15 @@ bool pkgDPkgPM::GoNoABIBreak(APT::Progress::PackageManager *progress)
    fd_set rfds;
    struct timespec tv;
 
-   unsigned int const MaxArgs = _config->FindI("Dpkg::MaxArgs",8*1024);
-   unsigned int const MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes",32*1024);
+   // FIXME: do we really need this limit when we have MaxArgBytes?
+   unsigned int const MaxArgs = _config->FindI("Dpkg::MaxArgs",32*1024);
+
+   // try to figure out the max environment size
+   int OSArgMax = sysconf(_SC_ARG_MAX);
+   if(OSArgMax < 0)
+      OSArgMax = 32*1024;
+   OSArgMax -= EnvironmentSize() - 2*1024;
+   unsigned int const MaxArgBytes = _config->FindI("Dpkg::MaxArgBytes", OSArgMax);
    bool const NoTriggers = _config->FindB("DPkg::NoTriggers", false);
 
    if (RunScripts("DPkg::Pre-Invoke") == false)
@@ -1413,22 +1502,8 @@ bool pkgDPkgPM::GoNoABIBreak(APT::Progress::PackageManager *progress)
       pid_t Child = ExecFork(KeepFDs);
       if (Child == 0)
       {
-         // This is the child
-	 if(d->slave >= 0 && d->master >= 0) 
-	 {
-	    setsid();
-	    int res = ioctl(d->slave, TIOCSCTTY, 0);
-            if (res < 0) {
-               std::cerr << "ioctl(TIOCSCTTY) failed for fd: " 
-                         << d->slave << std::endl;
-            } else {
-               close(d->master);
-               dup2(d->slave, 0);
-               dup2(d->slave, 1);
-               dup2(d->slave, 2);
-               close(d->slave);
-            }
-	 }
+	 // This is the child
+	 SetupSlavePtyMagic();
 	 close(fd[0]); // close the read end of the pipe
 
 	 dpkgChrootDirectory();
@@ -1668,9 +1743,10 @@ void pkgDPkgPM::WriteApportReport(const char *pkgpath, const char *errormsg)
    // do not report dpkg I/O errors, this is a format string, so we compare
    // the prefix and the suffix of the error with the dpkg error message
    vector<string> io_errors;
-   io_errors.push_back(string("failed to read on buffer copy for %s"));
-   io_errors.push_back(string("failed in write on buffer copy for %s"));
-   io_errors.push_back(string("short read on buffer copy for %s"));
+   io_errors.push_back(string("failed to read"));
+   io_errors.push_back(string("failed to write"));
+   io_errors.push_back(string("failed to seek"));
+   io_errors.push_back(string("unexpected end of file or stream"));
 
    for (vector<string>::iterator I = io_errors.begin(); I != io_errors.end(); ++I)
    {

apt-pkg/deb/dpkgpm.h

@@ -110,6 +110,7 @@ class pkgDPkgPM : public pkgPackageManager
    // helper
    void BuildPackagesProgressMap();
    void StartPtyMagic();
+   void SetupSlavePtyMagic();
    void StopPtyMagic();
    
    // input processing

apt-pkg/depcache.cc

@@ -663,10 +663,11 @@ void pkgDepCache::Update(OpProgress *Prog)
 {   
    iUsrSize = 0;
    iDownloadSize = 0;
-   iDelCount = 0;
    iInstCount = 0;
+   iDelCount = 0;
    iKeepCount = 0;
    iBrokenCount = 0;
+   iPolicyBrokenCount = 0;
    iBadCount = 0;
 
    // Perform the depends pass

apt-pkg/packagemanager.cc

@@ -372,6 +372,8 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg, int const Depth)
    std::list<DepIterator> needConfigure;
    do
    {
+      // Check each dependency and see if anything needs to be done
+      // so that it can be configured
       Changed = false;
       for (DepIterator D = instVer.DependsList(); D.end() == false; )
       {
@@ -383,7 +385,8 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg, int const Depth)
 	    continue;
 	 Bad = true;
 
-	 // Check for dependencies that have not been unpacked, probably due to loops.
+         // the first pass checks if we its all good, i.e. if we have
+         // to do anything at all
 	 for (DepIterator Cur = Start; true; ++Cur)
 	 {
 	    SPtrArray<Version *> VList = Cur.AllTargets();
@@ -402,6 +405,47 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg, int const Depth)
 		  break;
 	       }
 
+	       // Check if the version that is going to be installed will satisfy the dependency
+	       if (Cache[DepPkg].InstallVer != *I || List->IsNow(DepPkg) == false)
+		  continue;
+
+	       if (PkgLoop == true)
+	       {
+		  if (Debug)
+		     std::clog << OutputInDepth(Depth) << "Package " << Pkg << " loops in SmartConfigure" << std::endl;
+	          Bad = false;
+	       }
+	    }
+
+            if (Cur == End || Bad == false)
+	       break;
+         }
+
+         // this dependency is in a good state, so we can stop
+         if (Bad == false)
+         {
+            if (Debug)
+               std::clog << OutputInDepth(Depth) << "Found ok dep " << D.TargetPkg() << std::endl;
+            continue;
+         }
+
+	 // Check for dependencies that have not been unpacked, 
+         // probably due to loops.
+	 for (DepIterator Cur = Start; true; ++Cur)
+	 {
+	    SPtrArray<Version *> VList = Cur.AllTargets();
+
+	    for (Version **I = VList; *I != 0; ++I)
+	    {
+	       VerIterator Ver(Cache,*I);
+	       PkgIterator DepPkg = Ver.ParentPkg();
+
+	       // Check if the current version of the package is available and will satisfy this dependency
+	       if (DepPkg.CurrentVer() == Ver && List->IsNow(DepPkg) == true &&
+		   List->IsFlag(DepPkg,pkgOrderList::Removed) == false &&
+		   DepPkg.State() == PkgIterator::NeedsNothing)
+                  continue;
+
 	       // Check if the version that is going to be installed will satisfy the dependency
 	       if (Cache[DepPkg].InstallVer != *I || List->IsNow(DepPkg) == false)
 		  continue;
@@ -419,6 +463,8 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg, int const Depth)
 		  if (NonLoopingSmart(UNPACK_IMMEDIATE, Pkg, DepPkg, Depth, PkgLoop, &Bad, &Changed) == false)
 		     return false;
 	       }
+               // at this point we either unpacked a Dep or we are in a loop,
+               // no need to unpack a second one
 	       break;
 	    }
 
@@ -435,6 +481,7 @@ bool pkgPackageManager::SmartConfigure(PkgIterator Pkg, int const Depth)
          return _error->Error("Internal error: MaxLoopCount reached in SmartUnPack (1) for %s, aborting", Pkg.FullName().c_str());
    } while (Changed == true);
 
+   // now go over anything that needs configuring
    Bad = false, Changed = false, i = 0;
    do
    {

apt-pkg/tagfile.cc

@@ -526,7 +526,7 @@ static const char *iTFRewritePackageOrder[] = {
                           "Conffiles",
                           "Filename",
                           "Size",
-                          "MD5Sum",
+                          "MD5sum",
                           "SHA1",
                           "SHA256",
                           "SHA512",

apt-private/private-cacheset.cc

@@ -14,74 +14,77 @@
 
 #include <apti18n.h>
 
-bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile, 
-                                 LocalitySortedVersionSet &output_set,
-                                 OpProgress &progress)
+bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile,
+                                 APT::VersionContainerInterface * const vci,
+                                 OpProgress * const progress)
 {
     Matcher null_matcher = Matcher();
-    return GetLocalitySortedVersionSet(CacheFile, output_set, 
+    return GetLocalitySortedVersionSet(CacheFile, vci,
                                        null_matcher, progress);
 }
 
-bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile, 
-                                 LocalitySortedVersionSet &output_set,
+bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile,
+                                 APT::VersionContainerInterface * const vci,
                                  Matcher &matcher,
-                                 OpProgress &progress)
+                                 OpProgress * const progress)
 {
    pkgCache *Cache = CacheFile.GetPkgCache();
    pkgDepCache *DepCache = CacheFile.GetDepCache();
+   APT::CacheSetHelper helper(false);
 
    int Done=0;
-   progress.SubProgress(Cache->Head().PackageCount, _("Sorting"));
+   if (progress != NULL)
+      progress->SubProgress(Cache->Head().PackageCount, _("Sorting"));
+
+   bool const insertCurrentVer = _config->FindB("APT::Cmd::Installed", false);
+   bool const insertUpgradable = _config->FindB("APT::Cmd::Upgradable", false);
+   bool const insertManualInstalled = _config->FindB("APT::Cmd::Manual-Installed", false);
+
    for (pkgCache::PkgIterator P = Cache->PkgBegin(); P.end() == false; ++P)
    {
-      if (Done%500 == 0)
-         progress.Progress(Done);
-      Done++;
+      if (progress != NULL)
+      {
+	 if (Done % 500 == 0)
+	    progress->Progress(Done);
+	 ++Done;
+      }
+
+      // exclude virtual pkgs
+      if (P->VersionList == 0)
+	 continue;
 
       if ((matcher)(P) == false)
-         continue;
+	 continue;
 
-      // exclude virtual pkgs
-      if (P.VersionList() == 0)
-         continue;
       pkgDepCache::StateCache &state = (*DepCache)[P];
-      if (_config->FindB("APT::Cmd::Installed") == true)
+      if (insertCurrentVer == true)
       {
-         if (P.CurrentVer() != NULL)
-         {
-            output_set.insert(P.CurrentVer());
-         }
+	 if (P->CurrentVer != 0)
+	    vci->FromPackage(vci, CacheFile, P, APT::VersionContainerInterface::INSTALLED, helper);
       }
-      else if (_config->FindB("APT::Cmd::Upgradable") == true)
+      else if (insertUpgradable == true)
       {
-         if(P.CurrentVer() && state.Upgradable())
-         {
-             pkgPolicy *policy = CacheFile.GetPolicy();
-             output_set.insert(policy->GetCandidateVer(P));
-         }
+	 if(P.CurrentVer() && state.Upgradable())
+	    vci->FromPackage(vci, CacheFile, P, APT::VersionContainerInterface::CANDIDATE, helper);
       }
-      else if (_config->FindB("APT::Cmd::Manual-Installed") == true)
+      else if (insertManualInstalled == true)
       {
-         if (P.CurrentVer() && 
-             ((*DepCache)[P].Flags & pkgCache::Flag::Auto) == false)
-         {
-             pkgPolicy *policy = CacheFile.GetPolicy();
-             output_set.insert(policy->GetCandidateVer(P));
-         }
+	 if (P.CurrentVer() &&
+	       ((*DepCache)[P].Flags & pkgCache::Flag::Auto) == false)
+	    vci->FromPackage(vci, CacheFile, P, APT::VersionContainerInterface::CANDIDATE, helper);
       }
-      else 
+      else
       {
-         pkgPolicy *policy = CacheFile.GetPolicy();
-         if (policy->GetCandidateVer(P).IsGood())
-            output_set.insert(policy->GetCandidateVer(P));
-         else 
-            // no candidate, this may happen for packages in 
-            // dpkg "deinstall ok config-file" state - we pick the first ver
-            // (which should be the only one)
-            output_set.insert(P.VersionList());
+         if (vci->FromPackage(vci, CacheFile, P, APT::VersionContainerInterface::CANDIDATE, helper) == false)
+	 {
+	    // no candidate, this may happen for packages in
+	    // dpkg "deinstall ok config-file" state - we pick the first ver
+	    // (which should be the only one)
+	    vci->insert(P.VersionList());
+	 }
       }
    }
-   progress.Done();
+   if (progress != NULL)
+      progress->Done();
    return true;
 }

apt-private/private-cacheset.h

@@ -62,13 +62,13 @@ public:
 };
 
 // FIXME: add default argument for OpProgress (or overloaded function)
-bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile, 
-                                    LocalitySortedVersionSet &output_set,
+bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile,
+                                    APT::VersionContainerInterface * const vci,
                                     Matcher &matcher,
-                                    OpProgress &progress);
-bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile, 
-                                    LocalitySortedVersionSet &output_set,
-                                    OpProgress &progress);
+                                    OpProgress * const progress);
+bool GetLocalitySortedVersionSet(pkgCacheFile &CacheFile,
+                                    APT::VersionContainerInterface * const vci,
+                                    OpProgress * const progress);
 
 
 // CacheSetHelper saving virtual packages				/*{{{*/

apt-private/private-cmndline.cc

@@ -70,6 +70,8 @@ static bool addArgumentsAPTCache(std::vector<CommandLine::Args> &Args, char cons
    else
       return false;
 
+   bool const found_something = Args.empty() == false;
+
    // FIXME: move to the correct command(s)
    addArg('g', "generate", "APT::Cache::Generate", 0);
    addArg('t', "target-release", "APT::Default-Release", CommandLine::HasArg);
@@ -77,7 +79,8 @@ static bool addArgumentsAPTCache(std::vector<CommandLine::Args> &Args, char cons
 
    addArg('p', "pkg-cache", "Dir::Cache::pkgcache", CommandLine::HasArg);
    addArg('s', "src-cache", "Dir::Cache::srcpkgcache", CommandLine::HasArg);
-   return true;
+
+   return found_something;
 }
 									/*}}}*/
 static bool addArgumentsAPTCDROM(std::vector<CommandLine::Args> &Args, char const * const Cmd)/*{{{*/
@@ -172,6 +175,8 @@ static bool addArgumentsAPTGet(std::vector<CommandLine::Args> &Args, char const
       addArg('s', "no-act", "APT::Get::Simulate", 0);
    }
 
+   bool const found_something = Args.empty() == false;
+
    // FIXME: move to the correct command(s)
    addArg('d',"download-only","APT::Get::Download-Only",0);
    addArg('y',"yes","APT::Get::Assume-Yes",0);
@@ -197,7 +202,7 @@ static bool addArgumentsAPTGet(std::vector<CommandLine::Args> &Args, char const
    addArg(0,"install-suggests","APT::Install-Suggests",CommandLine::Boolean);
    addArg(0,"fix-policy","APT::Get::Fix-Policy-Broken",0);
 
-   return true;
+   return found_something;
 }
 									/*}}}*/
 static bool addArgumentsAPTMark(std::vector<CommandLine::Args> &Args, char const * const Cmd)/*{{{*/

apt-private/private-list.cc

@@ -85,15 +85,14 @@ private:
 };
 									/*}}}*/
 static void ListAllVersions(pkgCacheFile &CacheFile, pkgRecords &records,/*{{{*/
-                     pkgCache::PkgIterator P,    
-                     std::ostream &outs,
-                     bool include_summary=true)
+                     pkgCache::PkgIterator const &P, std::ostream &outs,
+                     std::string const &format)
 {
    for (pkgCache::VerIterator Ver = P.VersionList();
         Ver.end() == false; ++Ver)
    {
-      ListSingleVersion(CacheFile, records, Ver, outs, include_summary);
-      outs << "\n";
+      ListSingleVersion(CacheFile, records, Ver, outs, format);
+      outs << std::endl;
    }
 }
 									/*}}}*/
@@ -117,10 +116,9 @@ bool DoList(CommandLine &Cmd)
       patterns = Cmd.FileList + 1;
    }
 
-   std::map<std::string, std::string> output_map;
-   std::map<std::string, std::string>::const_iterator K;
-
-   bool includeSummary = _config->FindB("APT::Cmd::List-Include-Summary");
+   std::string format = "${color:highlight}${Package}${color:neutral}/${Origin} ${Version} ${Architecture}${ }${apt:Status}";
+   if (_config->FindB("APT::Cmd::List-Include-Summary", false) == true)
+      format += "\n  ${Description}\n";
 
    PackageNameMatcher matcher(patterns);
    LocalitySortedVersionSet bag;
@@ -129,37 +127,34 @@ bool DoList(CommandLine &Cmd)
                             Cache->Head().PackageCount, 
                             Cache->Head().PackageCount,
                             _("Listing"));
-   GetLocalitySortedVersionSet(CacheFile, bag, matcher, progress);
-   bool ShowAllVersions = _config->FindB("APT::Cmd::All-Versions", false);
+   GetLocalitySortedVersionSet(CacheFile, &bag, matcher, &progress);
+   bool const ShowAllVersions = _config->FindB("APT::Cmd::All-Versions", false);
+   std::map<std::string, std::string> output_map;
    for (LocalitySortedVersionSet::iterator V = bag.begin(); V != bag.end(); ++V)
    {
       std::stringstream outs;
       if(ShowAllVersions == true)
-      {
-         ListAllVersions(CacheFile, records, V.ParentPkg(), outs, includeSummary);
-         output_map.insert(std::make_pair<std::string, std::string>(
-            V.ParentPkg().Name(), outs.str()));
-      } else {
-         ListSingleVersion(CacheFile, records, V, outs, includeSummary);
-         output_map.insert(std::make_pair<std::string, std::string>(
-                           V.ParentPkg().Name(), outs.str()));
-      }
+         ListAllVersions(CacheFile, records, V.ParentPkg(), outs, format);
+      else
+         ListSingleVersion(CacheFile, records, V, outs, format);
+      output_map.insert(std::make_pair<std::string, std::string>(
+	       V.ParentPkg().Name(), outs.str()));
    }
 
    // FIXME: SORT! and make sorting flexible (alphabetic, by pkg status)
    // output the sorted map
+   std::map<std::string, std::string>::const_iterator K;
    for (K = output_map.begin(); K != output_map.end(); ++K)
       std::cout << (*K).second << std::endl;
 
-
    // be nice and tell the user if there is more to see
    if (bag.size() == 1 && ShowAllVersions == false)
    {
       // start with -1 as we already displayed one version
       int versions = -1;
       pkgCache::VerIterator Ver = *bag.begin();
-      for ( ; Ver.end() == false; Ver++)
-         versions++;
+      for ( ; Ver.end() == false; ++Ver)
+         ++versions;
       if (versions > 0)
          _error->Notice(P_("There is %i additional version. Please use the '-a' switch to see it", "There are %i additional versions. Please use the '-a' switch to see them.", versions), versions);
    }

apt-private/private-output.cc

@@ -118,7 +118,7 @@ static std::string GetFlagsStr(pkgCacheFile &CacheFile, pkgCache::PkgIterator P)
    std::string flags_str;
    if (state.NowBroken())
       flags_str = "B";
-   if (P.CurrentVer() && state.Upgradable())
+   if (P.CurrentVer() && state.Upgradable() && state.CandidateVer != NULL)
       flags_str = "g";
    else if (P.CurrentVer() != NULL)
       flags_str = "i";
@@ -164,15 +164,26 @@ static std::string GetVersion(pkgCacheFile &/*CacheFile*/, pkgCache::VerIterator
 									/*}}}*/
 static std::string GetArchitecture(pkgCacheFile &CacheFile, pkgCache::PkgIterator P)/*{{{*/
 {
-   pkgPolicy *policy = CacheFile.GetPolicy();
-   pkgCache::VerIterator inst = P.CurrentVer();
-   pkgCache::VerIterator cand = policy->GetCandidateVer(P);
-
-   // this may happen for packages in dpkg "deinstall ok config-file" state
-   if (inst.IsGood() == false && cand.IsGood() == false)
-      return P.VersionList().Arch();
-
-   return inst ? inst.Arch() : cand.Arch();
+   if (P->CurrentVer == 0)
+   {
+      pkgDepCache * const DepCache = CacheFile.GetDepCache();
+      pkgDepCache::StateCache const &state = (*DepCache)[P];
+      if (state.CandidateVer != NULL)
+      {
+	 pkgCache::VerIterator const CandV(CacheFile, state.CandidateVer);
+	 return CandV.Arch();
+      }
+      else
+      {
+	 pkgCache::VerIterator const V = P.VersionList();
+	 if (V.end() == false)
+	    return V.Arch();
+	 else
+	    return P.Arch();
+      }
+   }
+   else
+      return P.CurrentVer().Arch();
 }
 									/*}}}*/
 static std::string GetShortDescription(pkgCacheFile &CacheFile, pkgRecords &records, pkgCache::PkgIterator P)/*{{{*/
@@ -196,80 +207,90 @@ static std::string GetShortDescription(pkgCacheFile &CacheFile, pkgRecords &reco
    return ShortDescription;
 }
 									/*}}}*/
-void ListSingleVersion(pkgCacheFile &CacheFile, pkgRecords &records,	/*{{{*/
-                       pkgCache::VerIterator V, std::ostream &out,
-                       bool include_summary)
+static std::string GetLongDescription(pkgCacheFile &CacheFile, pkgRecords &records, pkgCache::PkgIterator P)/*{{{*/
 {
-   pkgCache::PkgIterator P = V.ParentPkg();
+   pkgPolicy *policy = CacheFile.GetPolicy();
 
-   pkgDepCache *DepCache = CacheFile.GetDepCache();
-   pkgDepCache::StateCache &state = (*DepCache)[P];
+   pkgCache::VerIterator ver;
+   if (P->CurrentVer != 0)
+      ver = P.CurrentVer();
+   else
+      ver = policy->GetCandidateVer(P);
 
-   std::string suite = GetArchiveSuite(CacheFile, V);
-   std::string name_str = P.Name();
+   std::string const EmptyDescription = "(none)";
+   if(ver.end() == true)
+      return EmptyDescription;
+
+   pkgCache::DescIterator const Desc = ver.TranslatedDescription();
+   pkgRecords::Parser & parser = records.Lookup(Desc.FileList());
+   std::string const longdesc = parser.LongDesc();
+   if (longdesc.empty() == true)
+      return EmptyDescription;
+   return SubstVar(longdesc, "\n ", "\n  ");
+}
+									/*}}}*/
+void ListSingleVersion(pkgCacheFile &CacheFile, pkgRecords &records,	/*{{{*/
+                       pkgCache::VerIterator const &V, std::ostream &out,
+                       std::string const &format)
+{
+   pkgCache::PkgIterator const P = V.ParentPkg();
+   pkgDepCache * const DepCache = CacheFile.GetDepCache();
+   pkgDepCache::StateCache const &state = (*DepCache)[P];
 
+   std::string output;
    if (_config->FindB("APT::Cmd::use-format", false))
+      output = _config->Find("APT::Cmd::format", "${db::Status-Abbrev} ${Package} ${Version} ${Origin} ${Description}");
+   else
+      output = format;
+
+   // FIXME: some of these names are really icky – and all is nowhere documented
+   output = SubstVar(output, "${db::Status-Abbrev}", GetFlagsStr(CacheFile, P));
+   output = SubstVar(output, "${Package}", P.Name());
+   std::string const ArchStr = GetArchitecture(CacheFile, P);
+   output = SubstVar(output, "${Architecture}", ArchStr);
+   std::string const InstalledVerStr = GetInstalledVersion(CacheFile, P);
+   output = SubstVar(output, "${installed:Version}", InstalledVerStr);
+   std::string const CandidateVerStr = GetCandidateVersion(CacheFile, P);
+   output = SubstVar(output, "${candidate:Version}", CandidateVerStr);
+   std::string const VersionStr = GetVersion(CacheFile, V);
+   output = SubstVar(output, "${Version}", VersionStr);
+   output = SubstVar(output, "${Origin}", GetArchiveSuite(CacheFile, V));
+
+   std::string StatusStr = "";
+   if (P->CurrentVer != 0)
    {
-      std::string format = _config->Find("APT::Cmd::format", "${db::Status-Abbrev} ${Package} ${Version} ${Origin} ${Description}");
-      std::string output = format;
-   
-      output = SubstVar(output, "${db::Status-Abbrev}", GetFlagsStr(CacheFile, P));
-      output = SubstVar(output, "${Package}", name_str);
-      output = SubstVar(output, "${installed:Version}", GetInstalledVersion(CacheFile, P));
-      output = SubstVar(output, "${candidate:Version}", GetCandidateVersion(CacheFile, P));
-      output = SubstVar(output, "${Version}", GetVersion(CacheFile, V));
-      output = SubstVar(output, "${Description}", GetShortDescription(CacheFile, records, P));
-      output = SubstVar(output, "${Origin}", GetArchiveSuite(CacheFile, V));
-      out << output << std::endl;
-   } else {
-      // raring/linux-kernel version [upradable: new-version]
-      //    description
-      pkgPolicy *policy = CacheFile.GetPolicy();
-      std::string VersionStr = GetVersion(CacheFile, V);
-      std::string CandidateVerStr = GetCandidateVersion(CacheFile, P);
-      std::string InstalledVerStr = GetInstalledVersion(CacheFile, P);
-      std::string StatusStr;
-      if(P.CurrentVer() == V && state.Upgradable()) {
-         strprintf(StatusStr, _("[installed,upgradable to: %s]"),
-                   CandidateVerStr.c_str());
-      } else if (P.CurrentVer() == V) {
-         if(!V.Downloadable())
-            StatusStr = _("[installed,local]");
-         else
-            if(V.Automatic() && state.Garbage)
-               StatusStr = _("[installed,auto-removable]");
-            else if (state.Flags & pkgCache::Flag::Auto)
-               StatusStr = _("[installed,automatic]");
-            else
-               StatusStr = _("[installed]");
-      } else if (P.CurrentVer() && 
-                 policy->GetCandidateVer(P) == V && 
-                 state.Upgradable()) {
-            strprintf(StatusStr, _("[upgradable from: %s]"),
-                      InstalledVerStr.c_str());
-      } else {
-         if (V.ParentPkg()->CurrentState == pkgCache::State::ConfigFiles)
-            StatusStr = _("[residual-config]");
-         else
-            StatusStr = "";
-      }
-      out << std::setiosflags(std::ios::left)
-          << _config->Find("APT::Color::Highlight", "")
-          << name_str 
-          << _config->Find("APT::Color::Neutral", "")
-          << "/" << suite
-          << " "
-          << VersionStr << " " 
-          << GetArchitecture(CacheFile, P);
-      if (StatusStr != "") 
-         out << " " << StatusStr;
-      if (include_summary)
+      if (P.CurrentVer() == V)
       {
-         out << std::endl 
-             << "  " << GetShortDescription(CacheFile, records, P)
-             << std::endl;
+	 if (state.Upgradable() && state.CandidateVer != NULL)
+	    strprintf(StatusStr, _("[installed,upgradable to: %s]"),
+		  CandidateVerStr.c_str());
+	 else if (V.Downloadable() == false)
+	    StatusStr = _("[installed,local]");
+	 else if(V.Automatic() == true && state.Garbage == true)
+	    StatusStr = _("[installed,auto-removable]");
+	 else if ((state.Flags & pkgCache::Flag::Auto) == pkgCache::Flag::Auto)
+	    StatusStr = _("[installed,automatic]");
+	 else
+	    StatusStr = _("[installed]");
       }
+      else if (state.CandidateVer == V && state.Upgradable())
+	 strprintf(StatusStr, _("[upgradable from: %s]"),
+	       InstalledVerStr.c_str());
    }
+   else if (V.ParentPkg()->CurrentState == pkgCache::State::ConfigFiles)
+      StatusStr = _("[residual-config]");
+   output = SubstVar(output, "${apt:Status}", StatusStr);
+   output = SubstVar(output, "${color:highlight}", _config->Find("APT::Color::Highlight", ""));
+   output = SubstVar(output, "${color:neutral}", _config->Find("APT::Color::Neutral", ""));
+   output = SubstVar(output, "${Description}", GetShortDescription(CacheFile, records, P));
+   output = SubstVar(output, "${LongDescription}", GetLongDescription(CacheFile, records, P));
+   output = SubstVar(output, "${ }${ }", "${ }");
+   output = SubstVar(output, "${ }\n", "\n");
+   output = SubstVar(output, "${ }", " ");
+   if (APT::String::Endswith(output, " ") == true)
+      output.erase(output.length() - 1);
+
+   out << output;
 }
 									/*}}}*/
 // ShowList - Show a list						/*{{{*/

apt-private/private-output.h

@@ -23,8 +23,8 @@ APT_PUBLIC extern unsigned int ScreenWidth;
 APT_PUBLIC bool InitOutput();
 
 void ListSingleVersion(pkgCacheFile &CacheFile, pkgRecords &records,
-                       pkgCache::VerIterator V, std::ostream &out,
-                       bool include_summary=true);
+                       pkgCache::VerIterator const &V, std::ostream &out,
+                       std::string const &format);
 
 
 // helper to describe global state

apt-private/private-search.cc

@@ -32,61 +32,95 @@ bool FullTextSearch(CommandLine &CmdL)					/*{{{*/
    pkgCacheFile CacheFile;
    pkgCache *Cache = CacheFile.GetPkgCache();
    pkgDepCache::Policy *Plcy = CacheFile.GetPolicy();
-   pkgRecords records(CacheFile);
    if (unlikely(Cache == NULL || Plcy == NULL))
       return false;
 
-   const char **patterns;
-   patterns = CmdL.FileList + 1;
+   // Make sure there is at least one argument
+   unsigned int const NumPatterns = CmdL.FileSize() -1;
+   if (NumPatterns < 1)
+      return _error->Error(_("You must give at least one search pattern"));
+
+#define APT_FREE_PATTERNS() for (std::vector<regex_t>::iterator P = Patterns.begin(); \
+      P != Patterns.end(); ++P) { regfree(&(*P)); }
+
+   // Compile the regex pattern
+   std::vector<regex_t> Patterns;
+   for (unsigned int I = 0; I != NumPatterns; ++I)
+   {
+      regex_t pattern;
+      if (regcomp(&pattern, CmdL.FileList[I + 1], REG_EXTENDED | REG_ICASE | REG_NOSUB) != 0)
+      {
+	 APT_FREE_PATTERNS();
+	 return _error->Error("Regex compilation error");
+      }
+      Patterns.push_back(pattern);
+   }
+
+   bool const NamesOnly = _config->FindB("APT::Cache::NamesOnly", false);
 
    std::map<std::string, std::string> output_map;
-   std::map<std::string, std::string>::const_iterator K;
 
    LocalitySortedVersionSet bag;
    OpTextProgress progress(*_config);
    progress.OverallProgress(0, 100, 50,  _("Sorting"));
-   GetLocalitySortedVersionSet(CacheFile, bag, progress);
+   GetLocalitySortedVersionSet(CacheFile, &bag, &progress);
    LocalitySortedVersionSet::iterator V = bag.begin();
 
    progress.OverallProgress(50, 100, 50,  _("Full Text Search"));
    progress.SubProgress(bag.size());
+   pkgRecords records(CacheFile);
+
+   std::string format = "${color:highlight}${Package}${color:neutral}/${Origin} ${Version} ${Architecture}${ }${apt:Status}\n";
+   if (_config->FindB("APT::Cache::ShowFull",false) == false)
+      format += "  ${Description}\n";
+   else
+      format += "  ${LongDescription}\n";
+
    int Done = 0;
+   std::vector<bool> PkgsDone(Cache->Head().PackageCount, false);
    for ( ;V != bag.end(); ++V)
    {
       if (Done%500 == 0)
          progress.Progress(Done);
       ++Done;
-      
-      int i;
+
+      // we want to list each package only once
+      pkgCache::PkgIterator const P = V.ParentPkg();
+      if (PkgsDone[P->ID] == true)
+	 continue;
+
+      char const * const PkgName = P.Name();
       pkgCache::DescIterator Desc = V.TranslatedDescription();
       pkgRecords::Parser &parser = records.Lookup(Desc.FileList());
-     
+      std::string const LongDesc = parser.LongDesc();
+
       bool all_found = true;
-      for(i=0; patterns[i] != NULL; ++i)
+      for (std::vector<regex_t>::const_iterator pattern = Patterns.begin();
+	    pattern != Patterns.end(); ++pattern)
       {
-         // FIXME: use regexp instead of simple find()
-         const char *pattern = patterns[i];
-         all_found &=  (
-            strstr(V.ParentPkg().Name(), pattern) != NULL ||
-            strcasestr(parser.ShortDesc().c_str(), pattern) != NULL ||
-            strcasestr(parser.LongDesc().c_str(), pattern) != NULL);
-         // search patterns are AND by default so we can skip looking further
-         // on the first mismatch
-         if(all_found == false)
-            break;
+	 if (regexec(&(*pattern), PkgName, 0, 0, 0) == 0)
+	    continue;
+	 else if (NamesOnly == false && regexec(&(*pattern), LongDesc.c_str(), 0, 0, 0) == 0)
+	    continue;
+	 // search patterns are AND, so one failing fails all
+	 all_found = false;
+	 break;
       }
-      if (all_found)
+      if (all_found == true)
       {
-            std::stringstream outs;
-            ListSingleVersion(CacheFile, records, V, outs);
-            output_map.insert(std::make_pair<std::string, std::string>(
-                                 V.ParentPkg().Name(), outs.str()));
+	 PkgsDone[P->ID] = true;
+	 std::stringstream outs;
+	 ListSingleVersion(CacheFile, records, V, outs, format);
+	 output_map.insert(std::make_pair<std::string, std::string>(
+		  PkgName, outs.str()));
       }
    }
+   APT_FREE_PATTERNS();
    progress.Done();
 
    // FIXME: SORT! and make sorting flexible (alphabetic, by pkg status)
    // output the sorted map
+   std::map<std::string, std::string>::const_iterator K;
    for (K = output_map.begin(); K != output_map.end(); ++K)
       std::cout << (*K).second << std::endl;
 

apt-private/private-update.cc

@@ -56,10 +56,17 @@ bool DoUpdate(CommandLine &CmdL)
       if (List->GetIndexes(&Fetcher,true) == false)
 	 return false;
 
+      std::string compExt = APT::Configuration::getCompressionTypes()[0];
       pkgAcquire::UriIterator I = Fetcher.UriBegin();
       for (; I != Fetcher.UriEnd(); ++I)
-	 c1out << '\'' << I->URI << "' " << flNotDir(I->Owner->DestFile) << ' ' << 
+      {
+         std::string FileName = flNotDir(I->Owner->DestFile);
+         if(compExt.empty() == false && 
+            APT::String::Endswith(FileName, compExt))
+            FileName = FileName.substr(0, FileName.size() - compExt.size() - 1);
+	 c1out << '\'' << I->URI << "' " << FileName << ' ' << 
             I->Owner->FileSize << ' ' << I->Owner->HashSum() << std::endl;
+      }
       return true;
    }
 
@@ -83,7 +90,7 @@ bool DoUpdate(CommandLine &CmdL)
       for (pkgCache::PkgIterator I = Cache->PkgBegin(); I.end() != true; ++I)
       {
          pkgDepCache::StateCache &state = Cache[I];
-         if (I->CurrentVer != 0 && state.Upgradable())
+         if (I->CurrentVer != 0 && state.Upgradable() && state.CandidateVer != NULL)
             upgradable++;
       }
       const char *msg = P_(

apt-private/private-utils.cc

@@ -8,45 +8,69 @@
 #include <cstdlib>
 #include <unistd.h>
 
-// DisplayFileInPager - Display File with pager        			/*{{{*/
-void DisplayFileInPager(std::string filename)
+// DisplayFileInPager - Display File with pager				/*{{{*/
+void DisplayFileInPager(std::string const &filename)
 {
-   std::string pager = _config->Find("Dir::Bin::Pager", 
-                                        "/usr/bin/sensible-pager");
-
    pid_t Process = ExecFork();
    if (Process == 0)
    {
       const char *Args[3];
-      Args[0] = pager.c_str();
       Args[1] = filename.c_str();
-      Args[2] = 0;
+      Args[2] = NULL;
+      if (isatty(STDOUT_FILENO) == 1)
+      {
+	 // likely installed, provided by sensible-utils
+	 std::string const pager = _config->Find("Dir::Bin::Pager",
+	       "sensible-pager");
+	 Args[0] = pager.c_str();
+	 execvp(Args[0],(char **)Args);
+	 // lets try some obvious alternatives
+	 Args[0] = getenv("PAGER");
+	 if (Args[0] != NULL)
+	    execvp(Args[0],(char **)Args);
+
+	 Args[0] = "pager";
+	 execvp(Args[0],(char **)Args);
+      }
+      // we could read the file ourselves, but… meh
+      Args[0] = "cat";
       execvp(Args[0],(char **)Args);
       exit(100);
    }
-         
+
    // Wait for the subprocess
-   ExecWait(Process, "sensible-pager", false);
+   ExecWait(Process, "pager", false);
 }
 									/*}}}*/
-// EditFileInSensibleEditor - Edit File with editor    			/*{{{*/
-void EditFileInSensibleEditor(std::string filename)
+// EditFileInSensibleEditor - Edit File with editor			/*{{{*/
+void EditFileInSensibleEditor(std::string const &filename)
 {
-   std::string editor = _config->Find("Dir::Bin::Editor", 
-                                        "/usr/bin/sensible-editor");
-
    pid_t Process = ExecFork();
    if (Process == 0)
    {
+      // likely installed, provided by sensible-utils
+      std::string const editor = _config->Find("Dir::Bin::Editor",
+	    "sensible-editor");
       const char *Args[3];
       Args[0] = editor.c_str();
       Args[1] = filename.c_str();
-      Args[2] = 0;
+      Args[2] = NULL;
+      execvp(Args[0],(char **)Args);
+      // the usual suspects we can try as an alternative
+      Args[0] = getenv("VISUAL");
+      if (Args[0] != NULL)
+	 execvp(Args[0],(char **)Args);
+
+      Args[0] = getenv("EDITOR");
+      if (Args[0] != NULL)
+	 execvp(Args[0],(char **)Args);
+
+      Args[0] = "editor";
       execvp(Args[0],(char **)Args);
       exit(100);
    }
-         
+
    // Wait for the subprocess
-   ExecWait(Process, "sensible-editor", false);
+   ExecWait(Process, "editor", false);
 }
 									/*}}}*/

apt-private/private-utils.h

@@ -5,7 +5,7 @@
 
 #include <string>
 
-APT_PUBLIC void DisplayFileInPager(std::string filename);
-APT_PUBLIC void EditFileInSensibleEditor(std::string filename);
+APT_PUBLIC void DisplayFileInPager(std::string const &filename);
+APT_PUBLIC void EditFileInSensibleEditor(std::string const &filename);
 
 #endif

buildlib/debiandoc.mak

@@ -1,60 +0,0 @@
-# -*- make -*-
-
-# This processes debian-doc sgml to produce html and plain text output
-
-# Input
-# $(SOURCE) - The documents to use
-
-# All output is writtin to files in the build doc directory
-
-# See defaults.mak for information about LOCAL
-
-# Some local definitions
-LOCAL := debiandoc-$(firstword $(SOURCE))
-$(LOCAL)-HTML := $(addsuffix .html,$(addprefix $(DOC)/,$(basename $(SOURCE))))
-$(LOCAL)-TEXT := $(addsuffix .text,$(addprefix $(DOC)/,$(basename $(SOURCE))))
-
-debiandoc:
-
-#---------
-
-# Rules to build HTML documentations
-ifdef DEBIANDOC_HTML
-
-# Install generation hooks
-debiandoc: $($(LOCAL)-HTML)
-veryclean: veryclean/html/$(LOCAL)
-
-vpath %.sgml $(SUBDIRS)
-$(DOC)/%.html: %.sgml
-	echo Creating html for $< to $@
-	-rm -rf $@
-	(HERE=`pwd`; cd $(@D) && $(DEBIANDOC_HTML) $(DEBIANDOC_HTML_OPTIONS) $$HERE/$<) || exit 199
-
-# Clean rule
-.PHONY: veryclean/html/$(LOCAL)
-veryclean/html/$(LOCAL):
-	-rm -rf $($(@F)-HTML)
-	
-endif
-
-#---------
-
-# Rules to build Text documentations
-ifdef DEBIANDOC_TEXT
-
-# Install generation hooks
-debiandoc: $($(LOCAL)-TEXT)
-veryclean: veryclean/text/$(LOCAL)
-
-vpath %.sgml $(SUBDIRS)
-$(DOC)/%.text: %.sgml
-	echo Creating text for $< to $@
-	$(DEBIANDOC_TEXT) -O $< > $@ || exit 198
-
-# Clean rule
-.PHONY: veryclean/text/$(LOCAL)
-veryclean/text/$(LOCAL):
-	-rm -rf $($(@F)-TEXT)
-	
-endif

buildlib/defaults.mak

@@ -76,7 +76,7 @@ PO_DOMAINS := $(BUILD)/po/domains
 
 # Module types
 LIBRARY_H = $(BASE)/buildlib/library.mak
-DEBIANDOC_H = $(BASE)/buildlib/debiandoc.mak
+DOCBOOK_H = $(BASE)/buildlib/docbook.mak
 MANPAGE_H = $(BASE)/buildlib/manpage.mak
 PROGRAM_H = $(BASE)/buildlib/program.mak
 PYTHON_H = $(BASE)/buildlib/python.mak
@@ -121,7 +121,7 @@ MKDIRS := $(BIN)
 all: dirs binary doc
 binary: library program
 maintainer-clean dist-clean distclean pristine sanity: veryclean
-startup headers library clean veryclean program test update-po manpages debiandoc:
+startup headers library clean veryclean program test update-po manpages docbook:
 
 veryclean:
 	echo Very Clean done for $(SUBDIR)

buildlib/docbook.mak

@@ -0,0 +1,75 @@
+# -*- make -*-
+
+# This processes DocBook XML to produce html and plain text output
+
+# Input
+# $(SOURCE) - The documents to use
+
+# All output is written to files in the build doc directory
+
+# See defaults.mak for information about LOCAL
+
+# Some local definitions
+LOCAL := docbook-$(firstword $(SOURCE))
+$(LOCAL)-HTML := $(addsuffix .html,$(addprefix $(DOC)/,$(basename $(SOURCE))))
+$(LOCAL)-TEXT := $(addsuffix .text,$(addprefix $(DOC)/,$(basename $(SOURCE))))
+INCLUDES = apt.ent apt-verbatim.ent apt-vendor.ent
+
+docbook:
+
+
+#---------
+
+# Rules to build HTML documentations
+ifdef XSLTPROC
+
+DOCBOOK_HTML_STYLESHEET := docbook-html-style.xsl
+
+# Install generation hooks
+docbook: $($(LOCAL)-HTML)
+veryclean: veryclean/html/$(LOCAL)
+
+vpath %.dbk $(SUBDIRS)
+vpath $(DOCBOOK_HTML_STYLESHEET) $(SUBDIRS)
+$(DOC)/%.html: %.dbk $(DOCBOOK_HTML_STYLESHEET) $(INCLUDES)
+	echo Creating html for $< to $@
+	-rm -rf $@
+	mkdir -p $@
+	$(DOCBOOK) \
+		--stringparam base.dir $@/ \
+		--stringparam l10n.gentext.default.language $(LC) \
+		$(<D)/$(DOCBOOK_HTML_STYLESHEET) $< || exit 199
+
+# Clean rule
+.PHONY: veryclean/html/$(LOCAL)
+veryclean/html/$(LOCAL):
+	-rm -rf $($(@F)-HTML)
+
+endif
+
+#---------
+
+# Rules to build Text documentations
+ifdef XSLTPROC
+
+DOCBOOK_TEXT_STYLESHEET := docbook-text-style.xsl
+
+# Install generation hooks
+docbook: $($(LOCAL)-TEXT)
+veryclean: veryclean/text/$(LOCAL)
+
+vpath %.dbk $(SUBDIRS)
+vpath $(DOCBOOK_TEXT_STYLESHEET) $(SUBDIRS)
+$(DOC)/%.text: %.dbk $(DOCBOOK_TEXT_STYLESHEET) $(INCLUDES)
+	echo Creating text for $< to $@
+	$(DOCBOOK) \
+		--stringparam l10n.gentext.default.language $(LC) \
+		$(<D)/$(DOCBOOK_TEXT_STYLESHEET) $< | \
+		LC_ALL=C.UTF-8 $(DOCBOOK2TEXT) > $@ || exit 198
+
+# Clean rule
+.PHONY: veryclean/text/$(LOCAL)
+veryclean/text/$(LOCAL):
+	-rm -rf $($(@F)-TEXT)
+
+endif

buildlib/environment.mak.in

@@ -28,15 +28,17 @@ RANLIB:=@RANLIB@
 GCC3DEP = @GCC3DEP@
 INLINEDEPFLAG = -MD
 
-# Debian doc stuff
-DEBIANDOC_HTML = @DEBIANDOC_HTML@
-DEBIANDOC_TEXT = @DEBIANDOC_TEXT@
-
 DOXYGEN = @DOXYGEN@
+W3M = @W3M@
 
-# xsltproc for the man pages
+# xsltproc for the man pages and documentation
 XSLTPROC := @XSLTPROC@
 
+# DocBook XML
+DOCBOOK = $(XSLTPROC) --nonet --novalid --xinclude
+DOCBOOK2TEXT = $(W3M) -o display_charset=UTF-8 -no-graph -T text/html \
+	-cols 78 -dump
+
 # po4a for the man pages
 PO4A := @PO4A@
 

buildlib/po4a_manpage.mak

@@ -15,6 +15,9 @@ INCLUDES = apt.ent apt-verbatim.ent apt-vendor.ent
 
 manpages:
 
+%.xsl: ../%.xsl
+	cp -a $< .
+
 # Do not use XMLTO, build the manpages directly with XSLTPROC
 ifdef XSLTPROC
 
@@ -34,13 +37,11 @@ apt-verbatim.ent: ../apt-verbatim.ent
 apt-vendor.ent: ../apt-vendor.ent
 	cp -a ../apt-vendor.ent .
 
-manpage-style.xsl: ../manpage-style.xsl
-	sed "/<!-- LANGUAGE -->/ i\
-<xsl:param name=\"l10n.gentext.default.language\" select=\"'$(LC)'\" />" ../manpage-style.xsl > manpage-style.xsl
-
 $($(LOCAL)-LIST) :: % : %.xml $(STYLESHEET) $(INCLUDES)
 	echo Creating man page $@
-	$(XSLTPROC) -o $@ $(STYLESHEET) $< || exit 200 # why xsltproc doesn't respect the -o flag here???
+	$(XSLTPROC) \
+		--stringparam l10n.gentext.default.language $(LC) \
+		-o $@ $(STYLESHEET) $< || exit 200 # why xsltproc doesn't respect the -o flag here???
 	test -f $(subst .$(LC),,$@) || echo 'FIXME: xsltproc respects the -o flag now, workaround can be removed'
 	mv -f $(subst .$(LC),,$@) $@
 
@@ -69,7 +70,6 @@ ifneq ($(words $(SOURCE)),0)
 include $(MANPAGE_H)
 endif
 
-# Debian Doc SGML Documents
-SOURCE := $(wildcard *.$(LC).sgml)
-DEBIANDOC_HTML_OPTIONS=-l $(LC).UTF-8
-include $(DEBIANDOC_H)
+# DocBook XML Documents
+SOURCE := $(wildcard *.$(LC).dbk)
+include $(DOCBOOK_H)

cmdline/apt-cache.cc

@@ -1278,8 +1278,8 @@ static bool Search(CommandLine &CmdL)
    ExDescFile *DFList = new ExDescFile[descCount];
    memset(DFList,0,sizeof(*DFList) * descCount);
 
-   bool PatternMatch[descCount * NumPatterns];
-   memset(PatternMatch,false,sizeof(PatternMatch));
+   bool *PatternMatch = new bool[descCount * NumPatterns];
+   memset(PatternMatch,false,sizeof(*PatternMatch) * descCount * NumPatterns);
 
    // Map versions that we want to write out onto the VerList array.
    for (pkgCache::GrpIterator G = Cache->GrpBegin(); G.end() == false; ++G)
@@ -1389,6 +1389,7 @@ static bool Search(CommandLine &CmdL)
    }
    
    delete [] DFList;
+   delete [] PatternMatch;
    for (unsigned I = 0; I != NumPatterns; I++)
       regfree(&Patterns[I]);
    if (ferror(stdout))

cmdline/apt-get.cc

@@ -665,7 +665,7 @@ static bool DoDownload(CommandLine &CmdL)
    {
       pkgAcquire::UriIterator I = Fetcher.UriBegin();
       for (; I != Fetcher.UriEnd(); ++I)
-	 cout << '\'' << I->URI << "' " << flNotDir(I->Owner->DestFile) << ' ' <<
+	 cout << '\'' << I->URI << "' " << flNotDir(I->Owner->DestFile)  << ' ' <<
 	       I->Owner->FileSize << ' ' << I->Owner->HashSum() << endl;
       return true;
    }
@@ -953,18 +953,19 @@ static bool DoSource(CommandLine &CmdL)
 	 else
 	 {
 	    // Call dpkg-source
-	    char S[500];
-	    snprintf(S,sizeof(S),"%s -x %s",
+	    std::string const sourceopts = _config->Find("DPkg::Source-Options", "-x");
+	    std::string S;
+	    strprintf(S, "%s %s %s",
 		     _config->Find("Dir::Bin::dpkg-source","dpkg-source").c_str(),
-		     Dsc[I].Dsc.c_str());
-	    if (system(S) != 0)
+		     sourceopts.c_str(), Dsc[I].Dsc.c_str());
+	    if (system(S.c_str()) != 0)
 	    {
-	       fprintf(stderr,_("Unpack command '%s' failed.\n"),S);
-	       fprintf(stderr,_("Check if the 'dpkg-dev' package is installed.\n"));
+	       fprintf(stderr, _("Unpack command '%s' failed.\n"), S.c_str());
+	       fprintf(stderr, _("Check if the 'dpkg-dev' package is installed.\n"));
 	       _exit(1);
-	    }	    
+	    }
 	 }
-	 
+
 	 // Try to compile it with dpkg-buildpackage
 	 if (_config->FindB("APT::Get::Compile",false) == true)
 	 {
@@ -980,20 +981,20 @@ static bool DoSource(CommandLine &CmdL)
 	    buildopts.append(_config->Find("DPkg::Build-Options","-b -uc"));
 
 	    // Call dpkg-buildpackage
-	    char S[500];
-	    snprintf(S,sizeof(S),"cd %s && %s %s",
+	    std::string S;
+	    strprintf(S, "cd %s && %s %s",
 		     Dir.c_str(),
 		     _config->Find("Dir::Bin::dpkg-buildpackage","dpkg-buildpackage").c_str(),
 		     buildopts.c_str());
-	    
-	    if (system(S) != 0)
+
+	    if (system(S.c_str()) != 0)
 	    {
-	       fprintf(stderr,_("Build command '%s' failed.\n"),S);
+	       fprintf(stderr, _("Build command '%s' failed.\n"), S.c_str());
 	       _exit(1);
-	    }	    
-	 }      
+	    }
+	 }
       }
-      
+
       _exit(0);
    }
 
@@ -1562,7 +1563,7 @@ static bool DoChangelog(CommandLine &CmdL)
    {
       string changelogfile;
       if (downOnly == false)
-	 changelogfile.append(tmpname).append("changelog");
+	 changelogfile.append(tmpname).append("/changelog");
       else
 	 changelogfile.append(Ver.ParentPkg().Name()).append(".changelog");
       if (DownloadChangelog(Cache, Fetcher, Ver, changelogfile) && downOnly == false)

cmdline/apt-helper.cc

@@ -16,6 +16,7 @@
 #include <apt-pkg/fileutl.h>
 #include <apt-pkg/acquire.h>
 #include <apt-pkg/acquire-item.h>
+#include <apt-pkg/proxy.h>
 
 #include <apt-private/acqprogress.h>
 #include <apt-private/private-output.h>
@@ -29,6 +30,19 @@
 #include <apti18n.h>
 									/*}}}*/
 
+static bool DoAutoDetectProxy(CommandLine &CmdL)
+{
+   if (CmdL.FileSize() != 2)
+      return _error->Error(_("Need one URL as argument"));
+   URI ServerURL(CmdL.FileList[1]);
+   AutoDetectProxy(ServerURL);
+   std::string SpecificProxy = _config->Find("Acquire::"+ServerURL.Access+"::Proxy::" + ServerURL.Host);
+   ioprintf(std::cout, "Using proxy '%s' for URL '%s'\n",
+            SpecificProxy.c_str(), std::string(ServerURL).c_str());
+
+   return true;
+}
+
 static bool DoDownloadFile(CommandLine &CmdL)
 {
    if (CmdL.FileSize() <= 2)
@@ -70,6 +84,7 @@ static bool ShowHelp(CommandLine &)
       "\n"
       "Commands:\n"
       "   download-file - download the given uri to the target-path\n"
+      "   auto-detect-proxy - detect proxy using apt.conf\n"
       "\n"
       "                       This APT helper has Super Meep Powers.\n");
    return true;
@@ -80,6 +95,7 @@ int main(int argc,const char *argv[])					/*{{{*/
 {
    CommandLine::Dispatch Cmds[] = {{"help",&ShowHelp},
 				   {"download-file", &DoDownloadFile},
+				   {"auto-detect-proxy", &DoAutoDetectProxy},
                                    {0,0}};
 
    std::vector<CommandLine::Args> Args = getCommandArgs(

configure.ac

@@ -18,7 +18,7 @@ AC_CONFIG_AUX_DIR(buildlib)
 AC_CONFIG_HEADER(include/config.h:buildlib/config.h.in include/apti18n.h:buildlib/apti18n.h.in)
 
 PACKAGE="apt"
-PACKAGE_VERSION="1.0.5"
+PACKAGE_VERSION="1.0.9.1"
 PACKAGE_MAIL="APT Development Team <deity@lists.debian.org>"
 AC_DEFINE_UNQUOTED(PACKAGE,"$PACKAGE")
 AC_DEFINE_UNQUOTED(PACKAGE_VERSION,"$PACKAGE_VERSION")
@@ -172,15 +172,12 @@ AC_EGREP_HEADER(h_errno, netdb.h, [AC_MSG_RESULT(normal)],
        [AC_MSG_ERROR("not found.")])
    ])
 
-dnl Check for debiandoc
-AC_PATH_PROG(DEBIANDOC_HTML,debiandoc2html)
-AC_PATH_PROG(DEBIANDOC_TEXT,debiandoc2text)
-
 dnl Check for doxygen
 AC_PATH_PROG(DOXYGEN, doxygen)
 
 dnl Check for the XSLTProc tool needed to build man pages together with po4a
 AC_PATH_PROG(XSLTPROC,xsltproc)
+AC_PATH_PROG(W3M, w3m)
 
 dnl Check for the po4a tool needed to build man pages
 AC_PATH_PROG(PO4A,po4a)

debian/changelog

@@ -1,3 +1,200 @@
+apt (1.0.9.3) unstable; urgency=medium
+
+  [ josch ]
+  * implement the updated build profile spec
+
+  [ Michael Vogt ]
+  * methods/rsh.cc: replace strcat with std::string (Closes: #76442)
+
+  [ Guillem Jover ]
+  * Update Status field values handling
+
+  [ David Kalnischkies ]
+  * don't cleanup cdrom files in apt-get update (Closes: 765458)
+
+ -- Michael Vogt <mvo@debian.org>  Wed, 15 Oct 2014 19:49:38 +0200
+
+apt (1.0.9.2) unstable; urgency=medium
+
+  [ Michael Vogt ]
+  * test/integration/test-apt-update-file: improve test
+  * Fix regression when copy: is used for a relative path (Closes: #762160)
+  * generalize Acquire::GzipIndex to support all compressions that
+    apt supports
+  * Fix regression for cdrom: sources from latest security update
+  * Ensure that iTFRewritePackageOrder is "MD5sum" to match 
+    apt-ftparchive
+  * debian/rules: add hardening=+all.
+    Thanks to Simon Ruderich, Markus Waldeck
+
+  [ Holger Wansing ]
+  * German program translation update (Closes: 762223)
+
+  [ Jérémy Bobbio ]
+  * disable timestamps in the footer of docs by doxygen
+
+  [ Trần Ngọc Quân ]
+  * Set STRIP_FROM_PATH for doxygen
+
+  [ Guillem Jover ]
+  * apt-get: Create the temporary downloaded changelog inside tmpdir
+    (closes: #763780) (CVE-2014-7206)
+
+ -- Michael Vogt <mvo@debian.org>  Thu, 02 Oct 2014 22:05:39 +0200
+
+apt (1.0.9.1) unstable; urgency=high
+
+  [ Michael Vogt ]
+  * Allow override of Proxy-Auto-Detect by the users configuration
+    (Closes: 759264)
+  * fix ci autopkgtest
+  * fix regression from 1.0.9 when file:/// source are used and
+    those are on a different partition than the apt state directory
+    and add regression test
+
+  [ Trần Ngọc Quân ]
+  * l10n: vi.po (636t): Update program translation
+
+  [ Chris Leick ]
+  * Updated German documentation translation
+
+  [ Mert Dirik ]
+  * Turkish program translation update (Closes: 761394)
+
+ -- Michael Vogt <mvo@debian.org>  Tue, 16 Sep 2014 20:52:25 +0200
+
+apt (1.0.9) unstable; urgency=high
+
+  * SECURITY UPDATE:
+    - incorrect invalidating of unauthenticated data (CVE-2014-0488)
+    - incorect verification of 304 reply (CVE-2014-0487)
+    - incorrect verification of Acquire::Gzip indexes (CVE-2014-0489)
+
+ -- Michael Vogt <mvo@debian.org>  Mon, 15 Sep 2014 08:34:46 +0200
+
+apt (1.0.8) unstable; urgency=medium
+
+  [ Holger Wansing ]
+  * German program translation update (Closes: 758837)
+
+  [ Américo Monteiro ]
+  * Portuguese manpages translation update (Closes: 759608)
+
+  [ Warren He ]
+  * initialize iPolicyBrokenCount in DepCache::Update (Closes: 758397)
+
+  [ Andreas Oberritter ]
+  * Avoid yielding blank lines with APT::Cmd::use-format=true
+
+  [ Michael Vogt ]
+  * Make Proxy-Auto-Detect check for each host (Closes: #759264)
+  * Add testcase for apt list --all-versions
+  * * apt-pkg/deb/dpkgpm.cc:
+    - update string matching for dpkg I/O errors. (LP: #1363257)
+    - properly parse the dpkg status line so that package name 
+      is properly set and an apport report is created. Thanks 
+      to Anders Kaseorg for the patch  (LP: #1353171)
+  * Use heap to allocate PatternMatch to avoid potential stack overflow
+    (Closes: 759612)
+  * Run autopkgtest tests with "env -i" to avoid pollution from the host env
+    (Closes: #759655)
+  * test/integration/test-ubuntu-bug-346386-apt-get-update-paywall: 
+    - use downloadfile() to fix test failure
+  * Fix incorrect upgradable listing in "apt list" 
+    (thanks to Michael Musenbrock) (Closes: #753297)
+  * apt-pkg/cachefile.cc:
+    - ensure we have a Policy in CacheFile.BuildDepCache()
+  * methods/http.cc:
+    - Improve Debug::Acquire::http debug output
+
+  [ Dimitri John Ledkov ]
+  * apt-ftparchive: make Packages & Sources generation optional, 
+    during Generate call
+
+  [ David Kalnischkies ]
+  * support regular expressions in 'apt search'
+  * implement --full in apt search
+  * fix progress report for upgrade and reinstall
+  * rework PTY magic to fix stair-stepping on kfreebsd (Closes: 759684)
+  * don't call pager in non-terminals for changelog (Closes: 755040)
+
+ -- Michael Vogt <mvo@debian.org>  Tue, 09 Sep 2014 20:09:11 +0200
+
+apt (1.0.7) unstable; urgency=medium
+
+  [ Michael Vogt ]
+  * add REAMDE.md
+  * StringToBool: only act if the entire string is consumed by strtol()
+  * Use @builddeps@ in the debian/tests/control file
+  * apt-pkg/acquire-item.cc: make pkgAcqDiffIndex more uniform
+  * Fix SmartConfigure to ignore ordering of packages that are already valid
+  * doc/apt.8.xml: fix typo, thanks to Jakub Wilk (Closes: #756056)
+  * doc/po/pt.po: updated, thanks to Américo Monteir (Closes: #756200)
+
+  [ victory ]
+  * Update Japanese documentation translation (Closes: #754817)
+
+  [ Trần Ngọc Quân ]
+  * l10n: vi.po (636t): Update one new string
+
+  [ Julian Andres Klode ]
+  * Fix debListParser to accept "no" as a value for the Multi-Arch field
+    (Closes: #759099)
+
+  [ Mert Dirik ]
+  * Turkish program translation update (Closes: 756710)
+
+  [ Miroslav Kure ]
+  * Czech program translation update (Closes: 758208)
+
+  [ David Kalnischkies ]
+  * add dpkg::source-options for dpkg-source invocation (Closes: 757534)
+  * support versioned provides as implemented by dpkg (Closes: 758153)
+
+ -- Michael Vogt <mvo@debian.org>  Wed, 27 Aug 2014 17:11:42 -0700
+
+apt (1.0.6) unstable; urgency=medium
+
+  [ Chris Leick ]
+  * German translation reviewed by Erik Pfannenstein
+
+  [ Michael Vogt ]
+  * methods/http.cc: use Req.str() in debug output
+  * Do not try to parse invalid translation files (LP: #756317)
+  * Do not clean "/" in pkgAcquire::Clean/pkgArchiveCleaner (Closes: #753531)
+  * Only show packages as upgradable if the have a CandidateVer != 0
+    (Closes: #753297)
+
+  [ Trần Ngọc Quân ]
+  * l10n: vi.po: Update 3 new messages
+
+  [ Joe Hansen ]
+  * Danish program translation update (Closes: 753979)
+
+  [ David Kalnischkies ]
+  * handle moved mmap after UniqFindTagWrite call (Closes: #753941)
+
+  [ Michele Orrù ]
+  * use printf instead of echo in testing framework
+
+  [ Cédric Barboiron ]
+  * Improve description how to turn off the caches (Closes: #753531)
+
+  [ Guillem Jover ]
+  * po: Fill or add missing Language field
+  * po: Remove fuzzy from file msgid header
+  * po: Fill Project-Id-Version with correct project id and version
+  * po: Fix Plural-Forms fields
+  * po: Fix or add missing email addresses
+  * po: Fix encoding issues
+  * po: Fix format specifier order in translation
+  * build: Set the XSL parameter through the command line instead of sed
+  * build: Convert from DebianDoc SGML to DocBook XML
+  * doc: Convert from DebianDoc SGML to DocBook XML
+  * doc: Unfuzzy DocBook translations
+
+ -- Michael Vogt <mvo@debian.org>  Thu, 10 Jul 2014 11:46:07 +0200
+
 apt (1.0.5) unstable; urgency=low
 
   [ Michael Vogt ]

debian/control

@@ -10,7 +10,7 @@ Build-Depends: dpkg-dev (>= 1.15.8), debhelper (>= 8.1.3~), libdb-dev,
  zlib1g-dev, libbz2-dev, liblzma-dev,
  xsltproc, docbook-xsl, docbook-xml, po4a (>= 0.34-2),
  autotools-dev, autoconf, automake, libgtest-dev
-Build-Depends-Indep: doxygen, debiandoc-sgml, graphviz
+Build-Depends-Indep: doxygen, w3m, graphviz
 Build-Conflicts: autoconf2.13, automake1.4
 Vcs-Git: git://anonscm.debian.org/apt/apt.git
 Vcs-Browser: http://anonscm.debian.org/gitweb/?p=apt/apt.git

debian/gbp.conf

@@ -3,3 +3,5 @@ prebuild = ./prepare-release pre-export
 postbuild = ./prepare-release post-build
 debian-branch = debian/sid
 debian-tag = %(version)s
+export-dir = ../build-area
+sign-tags = True

debian/rules

@@ -21,9 +21,12 @@ endif
 -include build/environment.mak
 
 ifneq (,$(shell which dpkg-buildflags))
-  export CXXFLAGS = $(shell dpkg-buildflags --get CXXFLAGS)
-  export LDFLAGS = $(shell dpkg-buildflags --get LDFLAGS)
-  export CPPFLAGS = $(shell dpkg-buildflags --get CPPFLAGS)
+  # make does not export to $(shell) so we need to workaround 
+  # (http://savannah.gnu.org/bugs/?10593)
+  dpkg_buildflags = DEB_BUILD_MAINT_OPTIONS=hardening=+all dpkg-buildflags
+  export CXXFLAGS = $(shell $(dpkg_buildflags) --get CXXFLAGS)
+  export LDFLAGS = $(shell $(dpkg_buildflags) --get LDFLAGS)
+  export CPPFLAGS = $(shell $(dpkg_buildflags) --get CPPFLAGS)
 else
   ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
     export CXXFLAGS = -O0 -g -Wall
@@ -70,7 +73,7 @@ LIBAPT_INST=libapt-inst$(LIBAPTINST_MAJOR)
 export DPKG_GENSYMBOLS_CHECK_LEVEL=0
 
 build-binary: build/build-binary-stamp
-build-debiandoc: build/build-debiandoc-stamp
+build-docbook: build/build-docbook-stamp
 build-manpages: build/build-manpages-stamp
 
 # Note that this is unconditionally done first as part of loading environment.mak
@@ -101,9 +104,9 @@ else
 endif
 	touch $@
 
-build/build-debiandoc-stamp: build/configure-stamp
+build/build-docbook-stamp: build/configure-stamp
 	# Add here commands to compile the package.
-	$(MAKE) debiandoc
+	$(MAKE) docbook
 	touch $@
 
 build/build-manpages-stamp: build/configure-stamp
@@ -126,7 +129,7 @@ debian/%.install: debian/%.install.in
 	sed 's/@DEB_HOST_MULTIARCH@/$(DEB_HOST_MULTIARCH)/g' $< > $@
 
 # Build architecture-independent files here.
-libapt-pkg-doc: build-debiandoc
+libapt-pkg-doc: build-docbook
 	dh_testdir -p$@
 	dh_testroot -p$@
 	dh_prep -p$@
@@ -153,7 +156,7 @@ libapt-pkg-doc: build-debiandoc
 	dh_md5sums -p$@
 	dh_builddeb -p$@
 
-apt-doc: build-debiandoc
+apt-doc: build-docbook
 	dh_testdir -p$@
 	dh_testroot -p$@
 	dh_prep -p$@
@@ -351,7 +354,7 @@ binary-arch: $(LIBAPT_PKG) $(LIBAPT_INST) apt libapt-pkg-dev apt-utils apt-trans
 binary-indep: apt-doc libapt-pkg-doc
 binary: binary-indep binary-arch
 build-arch: build-binary
-build-indep: build-manpages build-debiandoc
+build-indep: build-manpages build-docbook
 build: build-indep build-arch
 
 .PHONY: build clean binary-indep binary-arch binary

debian/tests/control

@@ -1,3 +1,3 @@
 Tests: run-tests
 Restrictions: allow-stderr 
-Depends: @, build-essential, fakeroot, wget, dpkg-dev, debhelper, libdb-dev, gettext, libcurl4-gnutls-dev, zlib1g-dev, libbz2-dev, xsltproc, docbook-xsl, docbook-xml, po4a, autotools-dev, autoconf, automake, doxygen, debiandoc-sgml, stunnel4, libdb-dev, db-util
+Depends: @, @builddeps@, fakeroot, wget, stunnel4, db-util

debian/tests/run-tests

@@ -8,7 +8,9 @@ if [ ! -e environment.mak ]; then
 fi
 make -C test/interactive-helper/
 
-# run against the installed apt
+# run tests against the installed apt, use "env -i" to ensure
+# the host env does not pollute our environment
+env -i \
 APT_INTEGRATION_TESTS_WEBSERVER_BIN_DIR=$(pwd)/build/bin \
 APT_INTEGRATION_TESTS_METHODS_DIR=/usr/lib/apt/methods \
 APT_INTEGRATION_TESTS_LIBEXEC_DIR=/usr/lib/apt/ \

doc/Bugs

@@ -1,174 +0,0 @@
-#24000: Bug in apt Version: 0.0.17-1bo0
- Summary: Couldn't locate an archive source for a package
- Status: Require Status file. 
-#24717: apt dies early if one postinst/preinst dies
- Status: Requires dpkg modifications
-#25104: APT should retry failed downloads
- Summary: FTP method has problems with busy servers
- Status: The 0.3.0 ftp method should probably use the configuration mechanism
-         to control this desired behavoir.
-#25176: Problem with FTP/firewall
-#25458: Problem with FTP/firewall
- Summary: FTP method has no controls for firewalls
- Status: The 0.3.0 ftp method should probably use the configuration mechanism
-         to control this desired behavoir.
- 
-#25026: apt: Why do you list the packages you're _not_ doing anything to instead of the ones you are?
-#22507: apt: apt-get - listing packages to be upgraded (wishlist)
- Summary: Feature request 
- Status: Fixed in 0.3.0 via the -u option
-#21113: [Apt] In dselect, update dies and exits if the packages file is not found.
- Status: Fixed in 0.3.0
-#22675: APT does not honor the --admin-dir option
-#22836: Disk/Zip-Method for apt?
- Summary: APT does not provide a way to download packages onto a 
-          removable media for another computer
- Status: 0.3.0 has substantially better support for this to the point
-         that it is doable by using a separate configuration file and
-	 the -c option
-#27601: srange errors from dselect
- Summary: Couldn't locate an archive source
- Status: Require status file
-         Believed to be fixed in 0.1.9, was not reproducible w/ given
-	 status file
-#27841: apt: apt depends on a missing library
- Status: New versions of APT in slink have been compiled with libstdc++2.9
-#23984: apt: support for "no_proxy" would be nice
- Status: Planned to be integrated into the new methods via the configuration
-         file
-	 Done - Use Acquire::http::proxy::host.com="DIRECT"
-#25021: apt: Need some control over multiple connections
- Status: Probable that 0.3.x will have support for configuing some
-         parameters
-	 Done - Use Acquire::Queue-Mode="access"
-#25019: apt: Confusing progress report
- Summary: Gripes about the progress meter
- Status: I do not intend to implement very many of these ideas in apt-get. 
-         The GUI will naturally be better
-	 I think the new progress meter address basically everyone's 
-	 concerns.
-#25022: apt: Lack of feedback on date checking
- Summary: Wants to know what package files were not updated
- Status: There is no place for this in the current apt-get design,
-         probably won't make the GUI either.
-	 Wee, the new acquire code allows this, it now prints out 'Hit'
-#26019: apt may report wrong line speed
-#26433: apt: claims to fetch things it doesn't actually fetch (wishlist)
-#28778: apt: apt's fetched message is wrong for update of packages files
- Summary: APT includes the fetch time from the local cache in its
-          calculations
- Status: Probably will be fixed with new acquire code
-         And it actually was fixed with the new acquire code
-#26670: apt: apt-get dumps core after checking integrity
- Summary: Some terminal environments cause APT to crash
-          Win95 telnet and emacs term-in-a-buffer are two at least
- Status: I have no idea why.
-         Seems to be gone in .3, whatever it was...
-#25001: apt: cleaned out archive even though not all files were installed
- Status: Apparently in some cases APT can return a success code even 
-         though it failed. I'm paying very close attention to this in
-	 0.3.x. Not to mention that the clean behavior will be 
-	 configurable..
-#28391: apt-get install without upgrading
- Summary: Make install leave the package in the keep state if it is already
-          installed
- Status: Will be implemented in 0.3.0
-	 Try the --no-upgrade options
-#28373: apt package is missing information on ftp.conf
-#29293: apt: Docs reference apt(8) but apt(8) not provided.
- Summary: The man pages have references to several non-existent items,
-          ftp.conf is only one of them.
- Status: Fix the man pages. This certainly will be done in 0.3.0
-#24799: Some suggestions for the apt method in dselect
- Summary: Wants to be able to specifiy -d from dselect
- Status: Likely a APT_OPTIONS environment variable will be created, -d can
-         be put there.
-	 There is already an APT_CONFIG in 0.3, APT_OPTIONS may also 
-	 appear..
-	 Use Dselect::Options "-d" and others
-#29920: Wish for more verbosity on "has no installation candidate"
- Summary: More reports
- Status: Already fixed in .3
-#29382: apt: apt deletes packages after installation without any question
- Summary: Obvois.
- Status: Fixed in .3, use Dselect::Clean "prompt"
-#30027: apt: version comparison bug
- Summary: Version compare differs from dpkg
- Status: Fixed in all CVS versions.
-#30260: apt: wishlist: do not return to main menu without prompting for return
- Summary: Wants to prompt after dselect update
- Status: Fixed in v3, use dselect::promptafterupdate "true";
-#29441: documenting the codes for -s
- Summary: Apparently the simulation output is not immediately obvois
- Status: Someone should update the man page.
-#24685: HTTP Proxy cache refresh
- Summary: Some caches hold onto the package index file for too long
- Status: It may be possible to insert the proper header to disable
-         caching but I can't think how to do this while allowing
-	 the cache to return cached objects - in effect it would
-	 completely disable the cache which may not be desired.
-         APT now sends a max age header. See the apt.conf(5)
-#28172: HTTP Proxy cache refresh should be forced for corrupted packages
- Summary: Some problem resulted in a corrupted package
- Status: I believe this reflects a deeper problem and the suggested solution
-         is only a band-aide patch. I intend to close this bug when #24685
-	 is fixed with a configuration directive.
-	 Use -o acquire::http::no-cache=true
-#29351: poor error message after conffile update
- Summary: APT errors when the package file is not found
- Status: The new APT warns better and the error is not fatal
-#30112: apt: internal error in apt
- Summary: APT gives an unclear error when it cannot correct dependencies
- Status: Error is clearer
-#30324: apt-get lying about file dates in /var/state/apt/lists
- Summary: Doesn't copy the file dates right with file URIs
- Status: Fixed in v3
-#30383: apt: dist-upgrade msg "E: Internal error, ScoredFix generated breaks."
- Summary: libc6 maddness causes bizzar problems
- Status: Fixed in v3 
-#29983: apt: Wrong diagnostic, could be better
- Summary: It doesn't explicy say you should be root
- Status: Fixed in v3
-#22892: Apt improvements
- Summary: Bails if an index file can't be found
- Status: Fixed in v3
-#28184: apt could be smarter regarding mirrors
- Summary: Make use of redundant sources
- Status: 0.3.0 will likely do this, already the information for it is stored.
-#27646: Apt: dpkg --merge-avail
- Summary: Suggestion to call merge avail after each update operation
- Status: Unlikely. The dpkg --print-avail functions should be obsoleted
-         by the apt-query program which should be written.
-         Use the dselect script, alias or something.
-#26663: rsync file access
- Status: Unlikely, rsync does not provide the necessary support to be
-         a terribly good method. I do not intend any 'split mode' 
-	 support (ie a way to get Package files via rsync and .debs
-	 via http) 
-#27100: apt: Better support for project/experimental
- Status: GUI Feature - Version selection and pinning
-         Honors the NotAutomatic flag
-#30643: base: apt-get update problems with http-/ftp-caches (i.e. squid)
- Status: Fixed as best I can. Try the Acquire::http::No-Cache option.
-
-#26592: apt: Problems with ftpd in SunOS 5.6
-#29903: apt-get insists onto sending a SIZE command
- Summary: SunOS ftpd does not support the SIZE command
- Status: Probably not worth fixing
-#20723: Apt suggestion
- Summary: Package Grouping Mechanism
- Status: Who knows
-#22550: apt-get upgrade could configure packages earlier
- Status: Who knows
-#23934: apt-get source <package>
-#27190: apt: installing source packages (wishlist)
- Status: Impossible to do without an index file for all source archives.
-#22551: apt: wish: use dist X iff pkg does not exist in dist Y
- Status: GUI Feature - Version selection and pinning
-#30237: apt: 0.1.9 checks dependencies against packages.gz
- Summary: For packages with identical version numbers APT prefers
-          to use the archive index files
- Status: Yes it is sorta bad, but there is no reasonable solution.

doc/Doxyfile.in

@@ -152,7 +152,7 @@ FULL_PATH_NAMES        = YES
 # will be relative from the directory where doxygen is started.
 # This tag requires that the tag FULL_PATH_NAMES is set to YES.
 
-STRIP_FROM_PATH        =
+STRIP_FROM_PATH        = @top_srcdir@
 
 # The STRIP_FROM_INC_PATH tag can be used to strip a user-defined part of the
 # path mentioned in the documentation of a class, which tells the reader which
@@ -161,7 +161,7 @@ STRIP_FROM_PATH        =
 # specify the list of include paths that are normally passed to the compiler
 # using the -I flag.
 
-STRIP_FROM_INC_PATH    =
+STRIP_FROM_INC_PATH    = @top_srcdir@
 
 # If the SHORT_NAMES tag is set to YES, doxygen will generate much shorter (but
 # less readable) file names. This can be useful is your file systems doesn't
@@ -1128,7 +1128,7 @@ HTML_COLORSTYLE_GAMMA  = 80
 # The default value is: YES.
 # This tag requires that the tag GENERATE_HTML is set to YES.
 
-HTML_TIMESTAMP         = YES
+HTML_TIMESTAMP         = NO
 
 # If the HTML_DYNAMIC_SECTIONS tag is set to YES then the generated HTML
 # documentation will contain sections that can be hidden and shown after the

doc/apt-cache.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-cdrom.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-config.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-extracttemplates.1.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-ftparchive.1.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-get.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-key.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-mark.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-secure.8.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-sortpkgs.1.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/apt-verbatim.ent

@@ -225,7 +225,7 @@
 ">
 
 <!-- this will be updated by 'prepare-release' -->
-<!ENTITY apt-product-version "1.0.5">
+<!ENTITY apt-product-version "1.0.9.1">
 
 <!-- (Code)names for various things used all over the place -->
 <!ENTITY oldstable-codename "squeeze">

doc/apt.8.xml

@@ -1,13 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>
@@ -105,8 +101,8 @@
      <listitem><para><literal>upgrade</literal> is used to install the
      newest versions of all packages currently installed on the system
      from the sources enumerated in
-     <filename>/etc/apt/sources.list</filename>. New package will be
-     installed, but existing package will never removed.
+     <filename>/etc/apt/sources.list</filename>. New packages will be
+     installed, but existing packages will never be removed.
      </para></listitem>
      </varlistentry>
 

doc/apt.conf.5.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>
@@ -608,10 +602,11 @@ DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt";};
    information, such as the two package caches <literal>srcpkgcache</literal> and 
    <literal>pkgcache</literal> as well as the location to place downloaded archives, 
    <literal>Dir::Cache::archives</literal>. Generation of caches can be turned off
-   by setting their names to the empty string. This will slow down startup but
-   save disk space. It is probably preferable to turn off the pkgcache rather
-   than the srcpkgcache. Like <literal>Dir::State</literal> the default
-   directory is contained in <literal>Dir::Cache</literal></para>
+   by setting <literal>pkgcache</literal> or <literal>srcpkgcache</literal> to
+   <literal>""</literal>.  This will slow down startup but save disk space. It
+   is probably preferable to turn off the pkgcache rather than the srcpkgcache.
+   Like <literal>Dir::State</literal> the default directory is contained in
+   <literal>Dir::Cache</literal></para>
 
    <para><literal>Dir::Etc</literal> contains the location of configuration files, 
    <literal>sourcelist</literal> gives the location of the sourcelist and 

doc/apt_preferences.5.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

doc/design.dbk

@@ -0,0 +1,439 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
+<title>The APT project design document</title>
+
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Manoj Srivastava</personname><email>srivasta@debian.org</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
+
+<abstract>
+<para>
+This document is an overview of the specifications and design goals of the APT
+project. It also attempts to give a broad description of the implementation
+as well.
+</para>
+</abstract>
+
+<copyright><year>1997</year><holder>Manoj Srivastava</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
+APT, including this document, is free software; you may redistribute it and/or
+modify it under the terms of the GNU General Public License as published by the
+Free Software Foundation; either version 2, or (at your option) any later
+version.
+</para>
+<para>
+This is distributed in the hope that it will be useful, but <emphasis>without
+any warranty</emphasis>; without even the implied warranty of merchantability
+or fitness for a particular purpose. See the GNU General Public License for
+more details.
+</para>
+<para>
+You should have received a copy of the GNU General Public License with your
+Debian system, in <literal>/usr/share/common-licenses/GPL</literal>, or with
+the <command>debiandoc-sgml</command> source package as the file
+<literal>COPYING</literal>. If not, write to the Free Software Foundation,
+Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+</para>
+</legalnotice>
+
+</bookinfo>
+
+<chapter id="introduction"><title>Introduction</title>
+<para>
+APT is supposed to be a replacement for dselect, and not a replacement for
+dpkg. However, since addition functionality has been required for APT, and
+given the fact that this is very closely related to dpkg, it is not
+unreasonable to expect that additional functionality in the underlying dpkg
+would also be requested.
+</para>
+<para>
+Deity/dselect are the first introduction that people have to Debian, and
+unfortunately this first impression contributes greatly to the public
+perception of the distribution. It is imperative that this be a showcase for
+Debian, rather than frighten novices away (which has been an accusation often
+levelled at the current system)
+</para>
+</chapter>
+
+<chapter id="ch2"><title>Requirements</title>
+<orderedlist numeration="arabic">
+<listitem>
+<para>
+APT should be a replacement for dselect. Therefore it should have all the
+functionality that dselect has currently. This is the primary means of
+interaction between the user and the package management system, and it should
+be able to handle all tasks involved in installing, upgrading, and routine
+management without having the users take recourse to the underlying management
+system.
+</para>
+</listitem>
+<listitem>
+<para>
+It should be easier to use and less confusing for novice users. The primary
+stimulus for the creation of APT was the perceived intractability, complexity,
+and non-intuitive behavior of the existing user interface, and as such, human
+factors must be a primary mandate of APT.
+</para>
+</listitem>
+<listitem>
+<para>
+It should be able to group packages more flexibly, and possibly allow
+operations based on a group. One should be able to select, or deselect,
+a coherent group of related packages simultaneously, allowing one to add,
+remove, or upgrade functionality to a machine as one step.
+</para>
+</listitem>
+<listitem>
+<para>
+This would allow APT to handle <emphasis>standard installations</emphasis>,
+namely, one could then install a set of packages to enable a machine to
+fulfill specific tasks. Define a few standard installations, and which
+packages are included therein. The packages should be internally consistent.
+</para>
+</listitem>
+<listitem>
+<para>
+Make use of a keywords field in package headers; provide a standard list of
+keywords for people to use. This could be the underpinning to allow the
+previous two requirements to work (though the developers are not constrained
+to implement the previous requirements using keywords)
+</para>
+</listitem>
+<listitem>
+<para>
+Use dependencies, conflicts, and reverse dependencies to properly order
+packages for installation and removal. This has been a complaint in the past
+that the installation methods do not really understand dependencies, causing
+the upgrade process to break, or allowing the removal of packages that left the
+system in an untenable state by breaking the dependencies on packages that were
+dependent on the package being removed. A special emphasis is placed on
+handling pre-dependencies correctly; the target of a predependency has to be
+fully configured before attempting to install the pre-dependent package. Also,
+<emphasis>configure immediately</emphasis> requests mentioned below should be
+handled.
+</para>
+</listitem>
+<listitem>
+<para>
+Handle replacement of a package providing a virtual package with another (for
+example, it has been very difficult replacing <command>sendmail</command> with
+<command>smail</command>, or vice versa), making sure that the dependencies are
+still satisfied.
+</para>
+</listitem>
+<listitem>
+<para>
+Handle source lists for updates from multiple sources. APT should also be able
+to handle diverse methods of acquiring new packages; local filesystem,
+mountable CD-ROM drives, FTP accessible repositories are some of the methods
+that come to mind. Also, the source lists can be separated into categories,
+such as main, contrib, non-us, non-local, non-free, my-very-own, etc. APT
+should be set up to retrieve the Packages files from these multiple source
+lists, as well as retrieving the packages themselves.
+</para>
+</listitem>
+<listitem>
+<para>
+Handle base of source and acquire all Packages files underneath. (possibly
+select based on architecture), this should be a simple extension of the
+previous requirement.
+</para>
+</listitem>
+<listitem>
+<para>
+Handle remote installation (to be implemented maybe in a future version, it
+still needs to be designed). This would ease the burden of maintaining
+multiple Debian machines on a site. In the authors opinion this is a killer
+difference for the distribution, though it may be too hard a problem to be
+implemented with the initial version of APT. However, some thought must be
+given to this to enable APT to retain hooks for future functionality, or at
+least to refrain from methods that may preclude remote activity. It is
+desirable that adding remote installation not require a redesign of APT from
+the ground up.
+</para>
+</listitem>
+<listitem>
+<para>
+Be scalable. Dselect worked a lot better with 400 packages, but at last count
+the number of packages was around twelve hundred and climbing. This also
+requires APT to pay attention to the needs of small machines which are low on
+memory (though this requirement shall diminish as we move towards bigger
+machines, it would still be nice if Debian worked on all old machines where
+Linux itself would work).
+</para>
+</listitem>
+<listitem>
+<para>
+Handle install immediately requests. Some packages, like watchdog, are
+required to be working for the stability of the machine itself. There are
+others which may be required for the correct functioning of a production
+machine, or which are mission critical applications. APT should, in these
+cases, upgrade the packages with minimal downtime; allowing these packages to
+be one of potentially hundreds of packages being upgraded concurrently may
+not satisfy the requirements of the package or the site. (Watchdog, for
+example, if not restarted quickly, may cause the machine to reboot in the
+midst of installation, which may cause havoc on the machine)
+</para>
+</listitem>
+</orderedlist>
+</chapter>
+
+<chapter id="ch3"><title>Procedural description</title>
+<variablelist>
+<varlistentry>
+<term>Set Options</term>
+<listitem>
+<para>
+This process handles setting of user or site options, and configuration of all
+aspects of APT. It allows the user to set the location and order of package
+sources, allowing them to set up source list details, like ftp site locations,
+passwords, etc. Display options may also be set.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Updates</term>
+<listitem>
+<para>
+Build a list of available packages, using source lists or a base location and
+trawling for Packages files (needs to be aware of architecture). This may
+involve finding and retrieving Packages files, storing them locally for
+efficiency, and parsing the data for later use. This would entail contacting
+various underlying access modules (ftp, cdrom mounts, etc) Use a backing store
+for speed. This may also require downloading the actual package files locally
+for speed.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Local status</term>
+<listitem>
+<para>
+Build up a list of packages already installed. This requires reading and
+writing the local??  status file. For remote installation, this should
+probably use similar mechanisms as the Packages file retrieval does. Use
+the backing store for speed. One should consider multiple backing stores,
+one for each machine.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Relationship determination</term>
+<listitem>
+<para>
+Determine forward and reverse dependencies. All known dependency fields should
+be acted upon, since it is fairly cheap to do so. Update the backing store
+with this information.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Selection</term>
+<listitem>
+<para>
+Present the data to the user. Look at Behan Webster's documentation for the
+user interface procedures. (Note: In the authors opinion deletions and reverse
+dependencies should also be presented to the user, in a strictly symmetric
+fashion; this may make it easier to prevent a package being removed that breaks
+dependencies)
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Ordering of package installations and configuration</term>
+<listitem>
+<para>
+Build a list of events. Simple topological sorting gives order of packages
+in dependency order. At certain points in this ordering,
+predependencies/immediate configure directives cause an break in normal
+ordering. We need to insert the uninstall/purge directive in the stream
+(default: as early as possible).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Action</term>
+<listitem>
+<para>
+Take the order of installations and removals and build up a stream of events
+to send to the packaging system (dpkg). Execute the list of events if
+successful. Do not partially install packages and leave system in broken
+state. Go to The Selection step as needed.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</chapter>
+
+<chapter id="ch4"><title>Modules and interfaces</title>
+<variablelist>
+<varlistentry>
+<term>The user interface module</term>
+<listitem>
+<para>
+Look at Behan Webster's documentation.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Widget set</term>
+<listitem>
+<para>
+Related closely to above Could some one present design decisions of the widget
+set here?
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>pdate Module</term>
+<listitem>
+<para>
+Distinct versions of the same package are recorded separately, but if multiple
+Packages files contain the same version of a package, then only the first one
+is recorded. For this reason, the least expensive update source should be
+listed first (local file system is better than a remote ftp site)
+</para>
+<para>
+This module should interact with the user interface module to set and change
+configuration parameters for the modules listed below. It needs to record that
+information in an on disk data file, to be read on future invocations.
+</para>
+<orderedlist numeration="arabic">
+<listitem>
+<para>
+FTP methods
+</para>
+</listitem>
+<listitem>
+<para>
+mount and file traversal module(s)?
+</para>
+</listitem>
+<listitem>
+<para>
+Other methods ???
+</para>
+</listitem>
+</orderedlist>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Status file parser/generator</term>
+<listitem>
+<para>
+The status file records the current state of the system, listing the packages
+installed, etc. The status file is also one method of communicating with dpkg,
+since it is perfectly permissible for the user to use APT to request packages
+be updated, put others on hold, mark other for removal, etc, and then run
+<literal>dpkg -BORGiE</literal> on a file system.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Package file parser/generator</term>
+<listitem>
+<para>
+Related to above. Handle multiple Packages files, from different
+sources. Each package contains a link back to the packages file structure
+that contains details about the origin of the data.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Dependency module</term>
+<listitem>
+<itemizedlist>
+<listitem>
+<para>
+dependency/conflict determination and linking
+</para>
+</listitem>
+<listitem>
+<para>
+reverse dependency generator. Maybe merged with above
+</para>
+</listitem>
+</itemizedlist>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Package ordering Module</term>
+<listitem>
+<para>
+Create an ordering of the actions to be taken.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Event generator</term>
+<listitem>
+<para>
+module to interact with dpkg
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</chapter>
+
+<chapter id="ch5"><title>Data flow and conversions analysis.</title>
+<screen>
+                                                          ____________
+                                                       __\|ftp modules|
+                                                      /  /|___________|
+                                    _ ____________   /     ________________
+                                    |    update   | /     |mount/local file|
+        |==========================&gt;|   module    |/_____\|  traversals    |
+        |                           |_____________|      /|________________|
+        |                             ^        ^
+        |                             |        |               ______________
+  ______|_______    _ _____ ______    |   _____v________      \|            |
+ |Configuration |   |configuration|   |   |Packages Files|  ===|Status file |
+ |  module      |&lt;=&gt;|    data     |   |   |______________| /  /|____________|
+ |______________|   |_____________|   |        ^          /
+         ^                            |        |         /
+         |                            | _______v_______|/_
+         |                            | |              |    ________________
+         |                            | |              |/_\|   Dependency  |
+         |                            | |backing store |\ /|     Module    |
+         |                            | |______________|  _|_______________|
+         |                             \       ^          /|       ^
+         |                              \      |         /         |
+         |                              _\|____v_______|/__    ____v_______
+         |_____________________________\| User interaction|    |    dpkg   |
+                                       /|_________________|&lt;==&gt;  Invoker  |
+                                                               |___________|
+</screen>
+<para>
+dpkg also interacts with status and available files.
+</para>
+<para>
+The backing store and the associated data structures are the core of APT. All
+modules essentially revolve around the backing store, feeding it data, adding
+and manipulating links and relationships between data in the backing store,
+allowing the user to interact with and modify the data in the backing store,
+and finally writing it out as the status file and possibly issuing directives
+to dpkg.
+</para>
+<para>
+The other focal point for APT is the user interface.
+</para>
+</chapter>
+
+</book>

doc/design.sgml

@@ -1,411 +0,0 @@
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<debiandoc>
-  <book>
-    <titlepag>
-      <title> The APT project design document</title>
-      <author>
-	<name>Manoj Srivastava</name>
-	<email>srivasta@debian.org</email>
-      </author>
-      <version>$Id: design.sgml,v 1.4 2003/02/12 15:05:45 doogie Exp $</version>
-      <abstract>
-	This document is an overview of the specifications and design
-	goals of the APT project. It also attempts to give a broad
-	description of the implementation as well.
-      </abstract>
-      <copyright>
-	<copyrightsummary>Copyright &copy;1997 Manoj Srivastava
-	</copyrightsummary>
-	<p>
-	  APT, including this document, is free software; you may
-	  redistribute it and/or modify it under the terms of the GNU
-	  General Public License as published by the Free Software
-	  Foundation; either version 2, or (at your option) any later
-	  version.</p>
-	<p>
-	  This is distributed in the hope that it will be useful, but
-	  <em>without any warranty</em>; without even the implied
-	  warranty of merchantability or fitness for a particular
-	  purpose.  See the GNU General Public License for more
-	  details.</p>
-
-	<p>
-	  You should have received a copy of the GNU General Public
-	  License with your Debian system, in
-	  <tt>/usr/share/common-licenses/GPL</tt>, or with the
-	  <prgn/debiandoc-sgml/ source package as the file
-	  <tt>COPYING</tt>.  If not, write to the Free Software
-	  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139,
-	  USA.</p>
-      </copyright>
-    </titlepag>
-    <chapt id="introduction">
-      <heading>Introduction</heading>
-      <p>APT is supposed to be a replacement for dselect, and not a 
-	replacement for dpkg. However, since addition functionality
-	has been required for APT, and given the fact that this is
-	very closely related to dpkg, it is not unreasonable to expect
-	that additional functionality in the underlying dpkg would
-	also be requested.</p>
-
-      <p> Deity/dselect are the first introduction that people have to
-	Debian, and unfortunately this first impression contributes
-	greatly to the public perception of the distribution. It is
-	imperative that this be a showcase for Debian, rather than
-	frighten novices away (which has been an accusation often
-	levelled at the current system)</p>
-    </chapt>
-    <chapt>
-      <heading>Requirements</heading>
-      <p>
-	<enumlist compact="compact">
-	  <item>
-	    <p>
-	      APT should be a replacement for dselect. Therefore it
-	      should have all the functionality that dselect has
-	      currently. This is the primary means of interaction
-	      between the user and the package management system, and
-	      it should be able to handle all tasks involved in
-	      installing, upgrading, and routine management without
-	      having the users take recourse to the underlying
-	      management system.</p>
-	  </item>
-	  <item>
-	    <p>
-	      It should be easier to use and less confusing for novice
-	      users. The primary stimulus for the creation of APT
-	      was the perceived intractability, complexity, and
-	      non-intuitive behavior of the existing user interface,
-	      and as such, human factors must be a primary mandate of
-	      APT.</p>
-	  </item>
-	  <item>
-	    <p>
-	      It should be able to group packages more flexibly, and
-	      possibly allow operations based on a group. One should
-	      be able to select, or deselect, a coherent group of
-	      related packages simultaneously, allowing one to add,
-	      remove, or upgrade functionality to a machine as one
-	      step.
-	    </p>
-	  </item>
-	  <item>
-	    <p>
-	      This would allow APT to handle <em>standard
-		installations</em>, namely, one could then install a
-	      set of packages to enable a machine to fulfill specific
-	      tasks.  Define a few standard installations, and which
-	      packages are included therein. The packages should be
-	      internally consistent.</p>
-	  </item>
-	  <item>
-	    <p>
-	      Make use of a keywords field in package headers; provide
-	      a standard list of keywords for people to use. This
-	      could be the underpinning to allow the previous two
-	      requirements to work (though the developers are not
-	      constrained to implement the previous requirements using
-	      keywords)
-	    </p>
-	  </item>
-	  <item>
-	    <p>
-	      Use dependencies, conflicts, and reverse dependencies to
-	      properly order packages for installation and
-	      removal. This has been a complaint in the past that the
-	      installation methods do not really understand
-	      dependencies, causing the upgrade process to break, or
-	      allowing the removal of packages that left the system in
-	      an untenable state by breaking the dependencies on
-	      packages that were dependent on the package being
-	      removed. A special emphasis is placed on handling
-	      pre-dependencies correctly; the target of a
-	      predependency has to be fully configured before
-	      attempting to install the pre-dependent package. Also,
-	      <em>configure immediately</em> requests mentioned below
-	      should be handled.</p>
-	  </item>
-	  <item>
-	    <p>
-	      Handle replacement of a package providing a virtual
-	      package with another (for example, it has been very
-	      difficult replacing <prgn>sendmail</prgn> with
-	      <prgn>smail</prgn>, or vice versa), making sure that the
-	      dependencies are still satisfied. </p>
-	  </item>
-	  <item>
-	    <p>
-	      Handle source lists for updates from multiple
-	      sources. APT should also be able to handle diverse
-	      methods of acquiring new packages; local filesystem,
-	      mountable CD-ROM drives, FTP accessible repositories are
-	      some of the methods that come to mind.  Also, the source
-	      lists can be separated into categories, such as main,
-	      contrib, non-us, non-local, non-free, my-very-own,
-	      etc. APT should be set up to retrieve the Packages
-	      files from these multiple source lists, as well as
-	      retrieving the packages themselves. </p>
-	  </item>
-	  <item>
-	    <p>
-	      Handle base of source and acquire all Packages files
-	      underneath.  (possibly select based on architecture),
-	      this should be a simple extension of the previous
-	      requirement.</p>
-	  </item>
-	  <item>
-	    <p>
-	      Handle remote installation (to be implemented maybe in a
-	      future version, it still needs to be designed). This
-	      would ease the burden of maintaining multiple Debian
-	      machines on a site. In the authors opinion this is a
-	      killer difference for the distribution, though it may be
-	      too hard a problem to be implemented with the initial
-	      version of APT. However, some thought must be given to
-	      this to enable APT to retain hooks for future
-	      functionality, or at least to refrain from methods that
-	      may preclude remote activity. It is desirable that
-	      adding remote installation not require a redesign of
-	      APT from the ground up.</p>
-	  </item>
-	  <item>
-	    <p>
-	      Be scalable. Dselect worked a lot better with 400
-	      packages, but at last count the number of packages was
-	      around twelve hundred and climbing. This also requires
-	      APT to pay attention to the needs of small machines
-	      which are low on memory (though this requirement shall
-	      diminish as we move towards bigger machines, it would
-	      still be nice if Debian worked on all old machines where
-	      Linux itself would work).</p>
-	  </item>
-	  <item>
-	    <p>
-	      Handle install immediately requests. Some packages, like
-	      watchdog, are required to be working for the stability
-	      of the machine itself. There are others which may be
-	      required for the correct functioning of a production
-	      machine, or which are mission critical
-	      applications. APT should, in these cases, upgrade the
-	      packages with minimal downtime; allowing these packages
-	      to be one of potentially hundreds of packages being
-	      upgraded concurrently may not satisfy the requirements
-	      of the package or the site. (Watchdog, for example, if
-	      not restarted quickly, may cause the machine to reboot
-	      in the midst of installation, which may cause havoc on
-	      the machine)</p>
-	  </item>
-	</enumlist> 
-      </p>
-    </chapt>
-    <chapt>
-      <heading>Procedural description</heading>
-      <p><taglist>
-	  <tag>Set Options</tag>
-	  <item>
-	    <p>
-	      This process handles setting of user or
-	      site options, and configuration of all aspects of
-	      APT. It allows the user to set the location and order
-	      of package sources, allowing them to set up source list
-	      details, like ftp site locations, passwords,
-	      etc. Display options may also be set.</p>
-	  </item>
-	  <tag>Updates</tag>
-	  <item>
-	    <p>
-	      Build a list of available packages, using
-	      source lists or a base location and trawling for
-	      Packages files (needs to be aware of architecture). This
-	      may involve finding and retrieving Packages files,
-	      storing them locally for efficiency, and parsing the
-	      data for later use. This would entail contacting various
-	      underlying access modules (ftp, cdrom mounts, etc) Use a
-	      backing store for speed. This may also require
-	      downloading the actual package files locally for
-	      speed.</p>
-	  </item>
-	  <tag>Local status</tag>
-	  <item>
-	    <p>
-	      Build up a list of packages already
-	      installed. This requires reading and writing the local??
-	      status file. For remote installation, this should
-	      probably use similar mechanisms as the Packages file
-	      retrieval does. Use the backing store for speed. One
-	      should consider multiple backing stores, one for each
-	      machine.
-	    </p>
-	  </item>
-	  <tag>Relationship determination</tag>
-	  <item>
-	    <p>
-	      Determine forward and reverse dependencies. All known
-	      dependency fields should be acted upon, since it is
-	      fairly cheap to do so. Update the backing store with
-	      this information.</p>
-	  </item>
-	  <tag>Selection</tag>
-	  <item>
-	    <p>
-	      Present the data to the user. Look at Behan Webster's
-	      documentation for the user interface procedures. (Note:
-	      In the authors opinion deletions and reverse
-	      dependencies should also be presented to the user, in a
-	      strictly symmetric fashion; this may make it easier to
-	      prevent a package being removed that breaks
-	      dependencies)
-	    </p>
-	  </item>
-	  <tag>Ordering of package installations and configuration </tag>
-	  <item>
-	    <p>
-	      Build a list of events. Simple topological sorting gives
-	      order of packages in dependency order. At certain points
-	      in this ordering, predependencies/immediate configure
-	      directives cause an break in normal ordering. We need to
-	      insert the uninstall/purge directive in the stream
-	      (default: as early as possible).</p>
-	  </item>
-	  <tag>Action</tag>
-	  <item>
-	    <p>
-	      Take the order of installations and removals and build
-	      up a stream of events to send to the packaging system
-	      (dpkg). Execute the list of events if successful. Do not
-	      partially install packages and leave system in broken
-	      state. Go to The Selection step as needed.</p>
-	  </item>
-
-	</taglist>
-      </p>
-    </chapt>
-    <chapt>
-      <heading>Modules and interfaces</heading>
-      <p><taglist>
-	  <tag>The user interface module</tag>
-	  <item>
-	    <p> Look at Behan Webster's documentation.</p> 
-	  </item>
-	  <tag>Widget set</tag>
-	  <item>
-	    <p>
-	      Related closely to above Could some one present design
-	      decisions of the widget set here?</p>
-	  </item>
-	  <tag>pdate Module</tag>
-	  <item>
-	    <p>	    
-	      Distinct versions of the same package are recorded
-	      separately, but if multiple Packages files contain the
-	      same version of a package, then only the first one is
-	      recorded. For this reason, the least expensive update
-	      source should be listed first (local file system is
-	      better than a remote ftp site)</p>
-	    <p>
-	      This module should interact with the user interface
-	      module to set and change configuration parameters for
-	      the modules listed below. It needs to record that
-	      information in an on disk data file, to be read on
-	      future invocations. </p>
-	    <p><enumlist>
-		<item>
-		  <p>FTP methods</p>
-		</item>
-		<item>
-		  <p>mount and file traversal module(s)?</p>
-		</item>
-		<item>
-		  <p>Other methods ???</p>
-		</item>
-	      </enumlist>
-	    </p>
-	  </item>
-	  <tag>Status file parser/generator</tag>
-	  <item>
-	    <p> 
-	      The status file records the current state of the system,
-	      listing the packages installed, etc. The status file is
-	      also one method of communicating with dpkg, since it is
-	      perfectly permissible for the user to use APT to
-	      request packages be updated, put others on hold, mark
-	      other for removal, etc, and then run <tt>dpkg
-		-BORGiE</tt> on a file system.</p>
-	  </item>
-	  <tag>Package file parser/generator</tag>
-	  <item>
-	    <p>
-	      Related to above. Handle multiple Packages files, from
-	      different sources. Each package contains a link back to
-	      the packages file structure that contains details about
-	      the origin of the data. </p>
-	  </item>
-	  <tag>Dependency module</tag>
-	  <item>
-	    <p><list>
-		<item>
-		  <p>dependency/conflict determination and linking</p>
-		</item>
-		<item>
-		  <p>reverse dependency generator. Maybe merged with above</p>
-		</item>
-	      </list>
-	    </p>
-	  </item>
-	  <tag>Package ordering Module</tag>
-	  <item>
-	    <p>Create an ordering of the actions to be taken.</p>
-	  </item>
-	  <tag>Event generator</tag>
-	  <item>
-	    <p>module to interact with dpkg</p>
-	  </item>
-	</taglist>
-    </chapt>
-    <chapt>
-      <heading>Data flow and conversions analysis.</heading>
-      <p> 
-	<example>
-                                                          ____________
-                                                       __\|ftp modules|
-                                                      /  /|___________|
-                                    _ ____________   /     ________________
-                                    |    update   | /     |mount/local file|
-        |==========================>|   module    |/_____\|  traversals    |
-        |                           |_____________|      /|________________|
-        |                             ^        ^
-        |                             |        |               ______________
-  ______|_______    _ _____ ______    |   _____v________      \|            |
- |Configuration |   |configuration|   |   |Packages Files|  ===|Status file |
- |  module      |<=>|    data     |   |   |______________| /  /|____________|
- |______________|   |_____________|   |        ^          /
-         ^                            |        |         /
-         |                            | _______v_______|/_
-         |                            | |              |    ________________
-         |                            | |              |/_\|   Dependency  |
-         |                            | |backing store |\ /|     Module    |
-         |                            | |______________|  _|_______________|
-         |                             \       ^          /|       ^
-         |                              \      |         /         |
-         |                              _\|____v_______|/__    ____v_______
-         |_____________________________\| User interaction|    |    dpkg   |
-                                       /|_________________|<==>|  Invoker  |
-                                                               |___________|
-
-	</example>
-      <p> dpkg also interacts with status and available files.</p>
-
-
-      <p>
-	The backing store and the associated data structures are the
-	core of APT. All modules essentially revolve around the
-	backing store, feeding it data, adding and manipulating links
-	and relationships between data in the backing store, allowing
-	the user to interact with and modify the data in the backing
-	store, and finally writing it out as the status file and
-	possibly issuing directives to dpkg.</p>
-
-      <p>The other focal point for APT is the user interface.</p> 
-    </chapt>
-  </book>
-</debiandoc>

doc/docbook-html-style.xsl

@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
+
+  <!-- Import our base stylesheet -->
+  <xsl:import href="/usr/share/xml/docbook/stylesheet/docbook-xsl/xhtml-1_1/chunk.xsl" />
+
+  <!-- Since we use xsltproc (not saxon), add a workaround to ensure UTF-8 -->
+  <xsl:template xmlns="http://www.w3.org/1999/xhtml" name="head.content.generator">
+    <xsl:param name="node" select="."/>
+    <meta name="generator" content="DocBook {$DistroTitle} V{$VERSION}"/>
+    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
+  </xsl:template>
+
+  <xsl:template name="generate.html.title"/>
+
+  <xsl:template match="releaseinfo" mode="titlepage.mode">
+    <xsl:apply-imports/>
+    <hr/>
+  </xsl:template>
+
+  <xsl:param name="root.filename">index</xsl:param>
+
+  <!-- We do not want a title in HTML. -->
+  <xsl:param name="generate.meta.abstract" select="0"/>
+
+  <!-- We do not want the first subsection on the same page as content. -->
+  <xsl:param name="chunk.first.sections" select="0"/>
+  <xsl:param name="chunk.section.depth" select="0"/>
+  <xsl:param name="chunker.output.indent" select="'yes'"/>
+
+  <xsl:param name="use.id.as.filename" select="1"/>
+
+  <xsl:param name="toc.section.depth" select="1"/>
+  <xsl:param name="generate.section.toc.level" select="0"/>
+  <xsl:param name="section.label.includes.component.label" select="1"/>
+  <xsl:param name="section.autolabel" select="1"/>
+
+  <xsl:param name="generate.css.header" select="1"/>
+
+</xsl:stylesheet>

doc/docbook-text-style.xsl

@@ -0,0 +1,70 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
+
+  <xsl:import href="/usr/share/xml/docbook/stylesheet/docbook-xsl/xhtml-1_1/docbook.xsl" />
+
+  <!-- Parameters for optimal text output. -->
+  <xsl:param name="callout.graphics" select="0"/>
+  <xsl:param name="callout.unicode" select="0"/>
+  <xsl:param name="section.autolabel" select="1"/>
+  <xsl:param name="section.label.includes.component.label" select="1"/>
+
+  <!-- Centering and aligning title elements. -->
+  <xsl:template match="/*/title[position()=1]" mode="titlepage.mode">
+    <br/>
+    <center>
+      <xsl:apply-imports/>
+    </center>
+    <br/>
+    <hr/> <!-- No underline, but at least something. -->
+  </xsl:template>
+  <xsl:template match="author|editor" mode="titlepage.mode">
+    <center>
+      <xsl:apply-imports/>
+    </center>
+  </xsl:template>
+
+  <xsl:template match="releaseinfo" mode="titlepage.mode">
+    <center>
+      <xsl:apply-imports/>
+    </center>
+    <hr/>
+  </xsl:template>
+
+  <!-- Dirty hack to get a left margin for paragraphs etc. -->
+  <xsl:template match="legalnotice/*
+        |chapter/*[not(name(.)='section') and not(name(.)='title')]
+        |section/*[not(name(.)='section') and not(name(.)='title')]
+        |appendix/*[not(name(.)='section') and not(name(.)='title')]
+        |footnote/*">
+    <xsl:copy><table><tr><td>&#xa0;&#xa0;&#xa0;</td><td>
+    <xsl:apply-imports/>
+    </td></tr></table></xsl:copy>
+  </xsl:template>
+
+  <!-- Skip URLs if it has something to print. -->
+  <xsl:template match="ulink[.!='']">
+    <xsl:copy-of select="."/>
+  </xsl:template>
+  <!-- Print URLs if nothing to print. -->
+  <xsl:template match="ulink[.='']">
+    <xsl:value-of select="@url"/>
+  </xsl:template>
+
+  <!-- Make clear where notes etc. begin and end. -->
+  <xsl:template match="caution|important|note|tip|warning">
+    <table width="80%" border="1">
+      <colgroup>
+        <col align="justify"/>
+      </colgroup>
+      <tbody>
+        <tr>
+          <td align="justify">
+            <xsl:apply-imports/>
+          </td>
+        </tr>
+      </tbody>
+    </table>
+  </xsl:template>
+
+</xsl:stylesheet>

doc/dpkg-tech.dbk

@@ -0,0 +1,878 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
+<title>dpkg technical manual</title>
+
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Tom Lees</personname><email>tom@lpsg.demon.co.uk</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
+
+<abstract>
+<para>
+This document describes the minimum necessary workings for the APT dselect
+replacement. It gives an overall specification of what its external interface
+must look like for compatibility, and also gives details of some internal
+quirks.
+</para>
+</abstract>
+
+<copyright><year>1997</year><holder>Tom Lees</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
+APT and this document are free software; you can redistribute them and/or
+modify them under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at your
+option) any later version.
+</para>
+<para>
+For more details, on Debian systems, see the file
+/usr/share/common-licenses/GPL for the full license.
+</para>
+</legalnotice>
+
+</bookinfo>
+
+<chapter id="ch1"><title>Quick summary of dpkg's external interface</title>
+
+<section id="control"><title>Control files</title>
+<para>
+The basic dpkg package control file supports the following major features:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+5 types of dependencies:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+Pre-Depends, which must be satisfied before a package may be unpacked
+</para>
+</listitem>
+<listitem>
+<para>
+Depends, which must be satisfied before a package may be configured
+</para>
+</listitem>
+<listitem>
+<para>
+Recommends, to specify a package which if not installed may severely limit the
+usefulness of the package
+</para>
+</listitem>
+<listitem>
+<para>
+Suggests, to specify a package which may increase the productivity of the
+package
+</para>
+</listitem>
+<listitem>
+<para>
+Conflicts, to specify a package which must NOT be installed in order for the
+package to be configured
+</para>
+</listitem>
+<listitem>
+<para>
+Breaks, to specify a package which is broken by the package and which should
+therefore not be configured while broken
+</para>
+</listitem>
+</itemizedlist>
+<para>
+Each of these dependencies can specify a version and a depedency on that
+version, for example "&lt;= 0.5-1", "== 2.7.2-1", etc. The comparators
+available are:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+"&lt;&lt;" - less than
+</para>
+</listitem>
+<listitem>
+<para>
+"&lt;=" - less than or equal to
+</para>
+</listitem>
+<listitem>
+<para>
+"&gt;&gt;" - greater than
+</para>
+</listitem>
+<listitem>
+<para>
+"&gt;=" - greater than or equal to
+</para>
+</listitem>
+<listitem>
+<para>
+"==" - equal to
+</para>
+</listitem>
+</itemizedlist>
+</listitem>
+<listitem>
+<para>
+The concept of "virtual packages", which many other packages may provide,
+using the Provides mechanism. An example of this is the "httpd" virtual
+package, which all web servers should provide. Virtual package names may be
+used in dependency headers. However, current policy is that virtual packages
+do not support version numbers, so dependencies on virtual packages with
+versions will always fail.
+</para>
+</listitem>
+<listitem>
+<para>
+Several other control fields, such as Package, Version, Description, Section,
+Priority, etc., which are mainly for classification purposes. The package
+name must consist entirely of lowercase characters, plus the characters '+',
+'-', and '.'. Fields can extend across multiple lines - on the second and
+subsequent lines, there is a space at the beginning instead of a field name
+and a ':'. Empty lines must consist of the text " .", which will be ignored,
+as will the initial space for other continuation lines. This feature is
+usually only used in the Description field.
+</para>
+</listitem>
+</itemizedlist>
+</section>
+
+<section id="s1.2"><title>The dpkg status area</title>
+<para>
+The "dpkg status area" is the term used to refer to the directory where dpkg
+keeps its various status files (GNU would have you call it the dpkg shared
+state directory). This is always, on Debian systems, /var/lib/dpkg. However,
+the default directory name should not be hard-coded, but #define'd, so that
+alteration is possible (it is available via configure in dpkg 1.4.0.9 and
+above). Of course, in a library, code should be allowed to override the
+default directory, but the default should be part of the library (so that
+the user may change the dpkg admin dir simply by replacing the library).
+</para>
+<para>
+Dpkg keeps a variety of files in its status area. These are discussed later
+on in this document, but a quick summary of the files is here:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+available - this file contains a concatenation of control information from all
+the packages which dpkg knows about. This is updated using the dpkg commands
+"--update-avail &lt;file&gt;", "--merge-avail &lt;file&gt;", and
+"--clear-avail".
+</para>
+</listitem>
+<listitem>
+<para>
+status - this file contains information on the following things for every
+package:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+Whether it is installed, not installed, unpacked, removed, failed
+configuration, or half-installed (deconfigured in favour of another package).
+</para>
+</listitem>
+<listitem>
+<para>
+Whether it is selected as install, hold, remove, or purge.
+</para>
+</listitem>
+<listitem>
+<para>
+If it is "ok" (no installation problems), or "not-ok".
+</para>
+</listitem>
+<listitem>
+<para>
+It usually also contains the section and priority (so that dselect may classify
+packages not in available)
+</para>
+</listitem>
+<listitem>
+<para>
+For packages which did not initially appear in the "available" file when they
+were installed, the other control information for them.
+</para>
+</listitem>
+</itemizedlist>
+<para>
+The exact format for the "Status:" field is:
+</para>
+<screen>
+      Status: Want Flag Status
+</screen>
+<para>
+Where <replaceable>Want</replaceable> may be one of
+<emphasis>unknown</emphasis>, <emphasis>install</emphasis>,
+<emphasis>hold</emphasis>, <emphasis>deinstall</emphasis>,
+<emphasis>purge</emphasis>. <replaceable>Flag</replaceable> may
+be one of <emphasis>ok</emphasis>, <emphasis>reinstreq</emphasis>.
+<replaceable>Status</replaceable> may
+be one of <emphasis>not-installed</emphasis>, <emphasis>config-files</emphasis>,
+<emphasis>half-installed</emphasis>, <emphasis>unpacked</emphasis>,
+<emphasis>half-configured</emphasis> and <emphasis>installed</emphasis>.
+The states are as follows:-
+</para>
+<variablelist>
+<varlistentry>
+<term>not-installed</term>
+<listitem>
+<para>
+No files are installed from the package, it has no config files left, it
+uninstalled cleanly if it ever was installed.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>unpacked</term>
+<listitem>
+<para>
+The basic files have been unpacked (and are listed in
+/var/lib/dpkg/info/[package].list. There are config files present, but the
+postinst script has _NOT_ been run.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>half-configured</term>
+<listitem>
+<para>
+The package was installed and unpacked, but the postinst script failed in some
+way.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>installed</term>
+<listitem>
+<para>
+All files for the package are installed, and the configuration was also
+successful.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>half-installed</term>
+<listitem>
+<para>
+An attempt was made to remove the packagem but there was a failure in the
+prerm script.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>config-files</term>
+<listitem>
+<para>
+The package was "removed", not "purged". The config files are left, but
+nothing else.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+<para>
+The two last items are only left in dpkg for compatibility - they are
+understood by it, but never written out in this form.
+</para>
+<para>
+Please see the dpkg source code, <literal>lib/parshelp.c</literal>,
+<emphasis>statusinfos</emphasis>, <emphasis>eflaginfos</emphasis> and
+<emphasis>wantinfos</emphasis> for more details.
+</para>
+</listitem>
+<listitem>
+<para>
+info - this directory contains files from the control archive of every
+package currently installed. They are installed with a prefix of
+"&lt;packagename&gt;.". In addition to this, it also contains a file
+called &lt;package&gt;.list for every package, which contains a list
+of files. Note also that the control file is not copied into here; it
+is instead found as part of status or available.
+</para>
+</listitem>
+<listitem>
+<para>
+methods - this directory is reserved for "method"-specific files - each
+"method" has a subdirectory underneath this directory (or at least,
+it can have). In addition, there is another subdirectory "mnt", where
+misc. filesystems (floppies, CD-ROMs, etc.) are mounted.
+</para>
+</listitem>
+<listitem>
+<para>
+alternatives - directory used by the "update-alternatives" program. It
+contains one file for each "alternatives" interface, which contains
+information about all the needed symlinked files for each alternative.
+</para>
+</listitem>
+<listitem>
+<para>
+diversions - file used by the "dpkg-divert" program. Each diversion takes
+three lines. The first is the package name (or ":" for user diversion), the
+second the original filename, and the third the diverted filename.
+</para>
+</listitem>
+<listitem>
+<para>
+updates - directory used internally by dpkg. This is discussed later, in the
+section <xref linkend="updates"/>.
+</para>
+</listitem>
+<listitem>
+<para>
+parts - temporary directory used by dpkg-split
+</para>
+</listitem>
+</itemizedlist>
+</section>
+
+<section id="s1.3"><title>The dpkg library files</title>
+<para>
+These files are installed under /usr/lib/dpkg (usually), but
+/usr/local/lib/dpkg is also a possibility (as Debian policy dictates). Under
+this directory, there is a "methods" subdirectory. The methods subdirectory in
+turn contains any number of subdirectories for each general method processor
+(note that one set of method scripts can, and is, used for more than one of
+the methods listed under dselect).
+</para>
+<para>
+The following files may be found in each of these subdirectories:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+names - One line per method, two-digit priority to appear on menu at
+beginning, followed by a space, the name, and then another space and
+the short description.
+</para>
+</listitem>
+<listitem>
+<para>
+desc.&lt;name&gt; - Contains the long description displayed by dselect
+when the cursor is put over the &lt;name&gt; method.
+</para>
+</listitem>
+<listitem>
+<para>
+setup - Script or program which sets up the initial values to be used
+by this method. Called with first argument as the status area directory
+(/var/lib/dpkg), second argument as the name of the method (as in the
+directory name), and the third argument as the option (as in the names file).
+</para>
+</listitem>
+<listitem>
+<para>
+install - Script/program called when the "install" option of dselect is run
+with this method. Same arguments as for setup.
+</para>
+</listitem>
+<listitem>
+<para>
+update - Script/program called when the "update" option of dselect is
+run. Same arguments as for setup/install.
+</para>
+</listitem>
+</itemizedlist>
+</section>
+
+<section id="s1.4"><title>The "dpkg" command-line utility</title>
+
+<section id="s1.4.1"><title>"Documented" command-line interfaces</title>
+<para>
+As yet unwritten. You can refer to the other manuals for now. See
+<citerefentry><refentrytitle>dpkg</refentrytitle><manvolnum>8</manvolnum></citerefentry>.
+</para>
+</section>
+
+<section id="s1.4.2"><title>Environment variables which dpkg responds to</title>
+<itemizedlist>
+<listitem>
+<para>
+DPKG_NO_TSTP - if set to a non-null value, this variable causes dpkg to run a
+child shell process instead of sending itself a SIGTSTP, when the user selects
+to background the dpkg process when it asks about conffiles.
+</para>
+</listitem>
+<listitem>
+<para>
+SHELL - used to determine which shell to run in the case when DPKG_NO_TSTP
+is set.
+</para>
+</listitem>
+<listitem>
+<para>
+CC - used as the C compiler to call to determine the target architecture. The
+default is "gcc".
+</para>
+</listitem>
+<listitem>
+<para>
+PATH - dpkg checks that it can find at least the following files in the path
+when it wants to run package installation scripts, and gives an error if it
+cannot find all of them:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+ldconfig
+</para>
+</listitem>
+<listitem>
+<para>
+start-stop-daemon
+</para>
+</listitem>
+<listitem>
+<para>
+install-info
+</para>
+</listitem>
+<listitem>
+<para>
+update-rc.d
+</para>
+</listitem>
+</itemizedlist>
+</listitem>
+</itemizedlist>
+</section>
+
+<section id="s1.4.3"><title>Assertions</title>
+<para>
+The dpkg utility itself is required for quite a number of packages, even if
+they have been installed with a tool totally separate from dpkg. The reason
+for this is that some packages, in their pre-installation scripts, check that
+your version of dpkg supports certain features. This was broken from the
+start, and it should have actually been a control file header "Dpkg-requires",
+or similar. What happens is that the configuration scripts will abort or
+continue according to the exit code of a call to dpkg, which will stop them
+from being wrongly configured.
+</para>
+<para>
+These special command-line options, which simply return as true or false are
+all prefixed with "--assert-". Here is a list of them (without the prefix):-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+support-predepends - Returns success or failure according to whether a version
+of dpkg which supports predepends properly (1.1.0 or above) is installed,
+according to the database.
+</para>
+</listitem>
+<listitem>
+<para>
+working-epoch - Return success or failure according to whether a version of
+dpkg which supports epochs in version properly (1.4.0.7 or above) is installed,
+according to the database.
+</para>
+</listitem>
+</itemizedlist>
+<para>
+Both these options check the status database to see what version of the
+"dpkg" package is installed, and check it against a known working version.
+</para>
+</section>
+
+<section id="s1.4.4"><title>--predep-package</title>
+<para>
+This strange option is described as follows in the source code:
+</para>
+<screen>
+/* Print a single package which:
+ *  (a) is the target of one or more relevant predependencies.
+ *  (b) has itself no unsatisfied pre-dependencies.
+ * If such a package is present output is the Packages file entry,
+ *  which can be massaged as appropriate.
+ * Exit status:
+ *  0 = a package printed, OK
+ *  1 = no suitable package available
+ *  2 = error
+ */
+</screen>
+<para>
+On further inspection of the source code, it appears that what is does is
+this:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+Looks at the packages in the database which are selected as "install",
+and are installed.
+</para>
+</listitem>
+<listitem>
+<para>
+It then looks at the Pre-Depends information for each of these packages
+from the available file. When it find a package for which any of the
+pre-dependencies are not satisfied, it breaks from the loop through the
+packages.
+</para>
+</listitem>
+<listitem>
+<para>
+It then looks through the unsatisfied pre-dependencies, and looks for
+packages which would satisfy this pre-dependency, stopping on the first
+it finds. If it finds none, it bombs out with an error.
+</para>
+</listitem>
+<listitem>
+<para>
+It then continues this for every dependency of the initial package.
+</para>
+</listitem>
+</itemizedlist>
+<para>
+Eventually, it writes out the record of all the packages to satisfy the
+pre-dependencies. This is used by the disk method to make sure that its
+dependency ordering is correct. What happens is that all pre-depending
+packages are first installed, then it runs dpkg -iGROEB on the directory,
+which installs in the order package files are found. Since pre-dependencies
+mean that a package may not even be unpacked unless they are satisfied, it
+is necessary to do this (usually, since all the package files are unpacked
+in one phase, the configured in another, this is not needed).
+</para>
+</section>
+
+</section>
+
+</chapter>
+
+<chapter id="ch2"><title>dpkg-deb and .deb file internals</title>
+<para>
+This chapter describes the internals to the "dpkg-deb" tool, which is used by
+"dpkg" as a back-end. dpkg-deb has its own tar extraction functions, which is
+the source of many problems, as it does not support long filenames, using
+extension blocks.
+</para>
+
+<section id="s2.1"><title>The .deb archive format</title>
+<para>
+The main principal of the new-format Debian archive (I won't describe the old
+format - for that have a look at deb-old.5), is that the archive really is an
+archive - as used by "ar" and friends. However, dpkg-deb uses this format
+internally, rather than calling "ar". Inside this archive, there are usually
+the following members:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+debian-binary
+</para>
+</listitem>
+<listitem>
+<para>
+control.tar.gz
+</para>
+</listitem>
+<listitem>
+<para>
+data.tar.gz
+</para>
+</listitem>
+</itemizedlist>
+<para>
+The debian-binary member consists simply of the string "2.0", indicating
+the format version. control.tar.gz contains the control files (and scripts),
+and the data.tar.gz contains the actual files to populate the filesystem
+with. Both tarfiles extract straight into the current directory. Information
+on the tar formats can be found in the GNU tar info page. Since dpkg-deb
+calls "tar -cf" to build packages, the Debian packages use the GNU extensions.
+</para>
+</section>
+
+<section id="s2.2"><title>The dpkg-deb command-line</title>
+<para>
+dpkg-deb documents itself thoroughly with its '--help' command-line
+option. However, I am including a reference to these for
+completeness. dpkg-deb supports the following options:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+--build (-b) &lt;dir&gt; - builds a .deb archive, takes a directory which
+contains all the files as an argument. Note that the directory
+&lt;dir&gt;/DEBIAN will be packed separately into the control archive.
+</para>
+</listitem>
+<listitem>
+<para>
+--contents (-c) &lt;debfile&gt; - Lists the contents of the "data.tar.gz"
+member.
+</para>
+</listitem>
+<listitem>
+<para>
+--control (-e) &lt;debfile&gt; - Extracts the control archive into a directory
+called DEBIAN. Alternatively, with another argument, it will extract it into a
+different directory.
+</para>
+</listitem>
+<listitem>
+<para>
+--info (-I) &lt;debfile&gt; - Prints the contents of the "control" file in the
+control archive to stdout. Alternatively, giving it other arguments will cause
+it to print the contents of those files instead.
+</para>
+</listitem>
+<listitem>
+<para>
+--field (-f) &lt;debfile&gt; &lt;field&gt; ... - Prints any number of fields
+from the "control" file. Giving it extra arguments limits the fields it prints
+to only those specified. With no command-line arguments other than a filename,
+it is equivalent to -I and just the .deb filename.
+</para>
+</listitem>
+<listitem>
+<para>
+--extract (-x) &lt;debfile&gt; &lt;dir&gt; - Extracts the data archive of a
+debian package under the directory &lt;dir&gt;.
+</para>
+</listitem>
+<listitem>
+<para>
+--vextract (-X) &lt;debfile&gt; &lt;dir&gt; - Same as --extract, except it
+is equivalent of giving tar the '-v' option - it prints the filenames as it
+extracts them.
+</para>
+</listitem>
+<listitem>
+<para>
+--fsys-tarfile &lt;debfile&gt; - This option outputs a gunzip'd version of
+data.tar.gz to stdout.
+</para>
+</listitem>
+<listitem>
+<para>
+--new - sets the archive format to be used to the new Debian format
+</para>
+</listitem>
+<listitem>
+<para>
+--old - sets the archive format to be used to the old Debian format
+</para>
+</listitem>
+<listitem>
+<para>
+--debug - Tells dpkg-deb to produce debugging output
+</para>
+</listitem>
+<listitem>
+<para>
+--nocheck - Tells dpkg-deb not to check the sanity of the control file
+</para>
+</listitem>
+<listitem>
+<para>
+--help (-h) - Gives a help message
+</para>
+</listitem>
+<listitem>
+<para>
+--version - Shows the version number
+</para>
+</listitem>
+<listitem>
+<para>
+--licence/--license (UK/US spellings) - Shows a brief outline of the GPL
+</para>
+</listitem>
+</itemizedlist>
+
+<section id="s2.2.1"><title>Internal checks used by dpkg-deb when building packages</title>
+<para>
+Here is a list of the internal checks used by dpkg-deb when building
+packages. It is in the order they are done.
+</para>
+<itemizedlist>
+<listitem>
+<para>
+First, the output Debian archive argument, if it is given, is checked using
+stat. If it is a directory, an internal flag is set. This check is only made
+if the archive name is specified explicitly on the command-line. If the
+argument was not given, the default is the directory name, with ".deb"
+appended.
+</para>
+</listitem>
+<listitem>
+<para>
+Next, the control file is checked, unless the --nocheck flag was specified on
+the command-line. dpkg-deb will bomb out if the second argument to --build was
+a directory, and --nocheck was specified. Note that dpkg-deb will not be able
+to determine the name of the package in this case. In the control file, the
+following things are checked:-
+</para>
+<itemizedlist>
+<listitem>
+<para>
+The package name is checked to see if it contains any invalid characters (see
+<xref linkend="control"/> for this).
+</para>
+</listitem>
+<listitem>
+<para>
+The priority field is checked to see if it uses standard values, and
+user-defined values are warned against. However, note that this check is now
+redundant, since the control file no longer contains the priority - the
+changes file now does this.
+</para>
+</listitem>
+<listitem>
+<para>
+The control file fields are then checked against the standard list of fields
+which appear in control files, and any "user-defined" fields are reported as
+warnings.
+</para>
+</listitem>
+<listitem>
+<para>
+dpkg-deb then checks that the control file contains a valid version number.
+</para>
+</listitem>
+</itemizedlist>
+</listitem>
+<listitem>
+<para>
+After this, in the case where a directory was specified to build the .deb file
+in, the filename is created as "directory/pkg_ver.deb" or
+"directory/pkg_ver_arch.deb", depending on whether the control file contains
+an architecture field.
+</para>
+</listitem>
+<listitem>
+<para>
+Next, dpkg-deb checks for the &lt;dir&gt;/DEBIAN directory. It complains if it
+doesn't exist, or if it has permissions &lt; 0755, or &gt; 0775.
+</para>
+</listitem>
+<listitem>
+<para>
+It then checks that all the files in this subdir are either symlinks or plain
+files, and have permissions between 0555 and 0775.
+</para>
+</listitem>
+<listitem>
+<para>
+The conffiles file is then checked to see if the filenames are too
+long. Warnings are produced for each that is. After this, it checks
+that the package provides initial copies of each of these conffiles,
+and that they are all plain files.
+</para>
+</listitem>
+</itemizedlist>
+</section>
+
+</section>
+
+</chapter>
+
+<chapter id="ch3"><title>dpkg internals</title>
+<para>
+This chapter describes the internals of dpkg itself. Although the low-level
+formats are quite simple, what dpkg does in certain cases often does not make
+sense.
+</para>
+
+<section id="updates"><title>Updates</title>
+<para>
+This describes the /var/lib/dpkg/updates directory. The function of this
+directory is somewhat strange, and seems only to be used internally. A
+function called cleanupdates is called whenever the database is scanned. This
+function in turn uses
+<citerefentry><refentrytitle>scandir</refentrytitle><manvolnum>3</manvolnum></citerefentry>,
+to sort the files in this directory. Files who names do not consist entirely
+of digits are discarded. dpkg also causes a fatal error if any of the
+filenames are different lengths.
+</para>
+<para>
+After having scanned the directory, dpkg in turn parses each file the same way
+it parses the status file (they are sorted by the scandir to be in numerical
+order). After having done this, it then writes the status information back to
+the "status" file, and removes all the "updates" files.
+</para>
+<para>
+These files are created internally by dpkg's "checkpoint" function, and are
+cleaned up when dpkg exits cleanly.
+</para>
+<para>
+Juding by the use of the updates directory I would call it a Journal. Inorder
+to efficiently ensure the complete integrity of the status file dpkg will
+"checkpoint" or journal all of it's activities in the updates directory. By
+merging the contents of the updates directory (in order!!) against the original
+status file it can get the precise current state of the system, even in the
+event of a system failure while dpkg is running.
+</para>
+<para>
+The other option would be to sync-rewrite the status file after each operation,
+which would kill performance.
+</para>
+<para>
+It is very important that any program that uses the status file abort if the
+updates directory is not empty!  The user should be informed to run dpkg
+manually (what options though??) to correct the situation.
+</para>
+</section>
+
+<section id="s3.2"><title>What happens when dpkg reads the database</title>
+<para>
+First, the status file is read. This gives dpkg an initial idea of the
+packages that are there. Next, the updates files are read in, overriding the
+status file, and if necessary, the status file is re-written, and updates files
+are removed. Finally, the available file is read. The available file is read
+with flags which preclude dpkg from updating any status information from it,
+though - installed version, etc., and is also told to record that the packages
+it reads this time are available, not installed.
+</para>
+<para>
+More information on updates is given above.
+</para>
+</section>
+
+<section id="s3.3"><title>How dpkg compares version numbers</title>
+<para>
+Version numbers consist of three parts: the epoch, the upstream version, and
+the Debian revision. Dpkg compares these parts in that order. If the epochs
+are different, it returns immediately, and so on.
+</para>
+<para>
+However, the important part is how it compares the versions which are
+essentially stored as just strings. These are compared in two distinct
+parts: those consisting of numerical characters (which are evaluated, and
+then compared), and those consisting of other characters. When comparing
+non-numerical parts, they are compared as the character values (ASCII),
+but non-alphabetical characters are considered "greater than" alphabetical
+ones. Also note that longer strings (after excluding differences where
+numerical values are equal) are considered "greater than" shorter ones.
+</para>
+<para>
+Here are a few examples of how these rules apply:-
+</para>
+<screen>
+15 &gt; 10
+0010 == 10
+
+d.r &gt; dsr
+32.d.r == 0032.d.r
+d.rnr &lt; d.rnrn
+</screen>
+</section>
+
+</chapter>
+
+</book>

doc/dpkg-tech.sgml

@@ -1,511 +0,0 @@
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<book>
-<title>dpkg technical manual</title>
-
-<author>Tom Lees <email>tom@lpsg.demon.co.uk</email></author>
-<version>$Id: dpkg-tech.sgml,v 1.3 2003/02/12 15:05:45 doogie Exp $</version>
-
-<abstract>
-This document describes the minimum necessary workings for the APT dselect
-replacement. It gives an overall specification of what its external interface
-must look like for compatibility, and also gives details of some internal
-quirks.
-</abstract>
-
-<copyright>
-Copyright &copy; Tom Lees, 1997.
-<p>
-APT and this document are free software; you can redistribute them and/or
-modify them under the terms of the GNU General Public License as published
-by the Free Software Foundation; either version 2 of the License, or (at your
-option) any later version.
-
-<p>
-For more details, on Debian systems, see the file
-/usr/share/common-licenses/GPL for the full license.
-</copyright>
-
-<toc sect>
-
-<chapt>Quick summary of dpkg's external interface
-<sect id="control">Control files
-
-<p>
-The basic dpkg package control file supports the following major features:-
-
-<list>
-<item>5 types of dependencies:-
-	<list>
-	<item>Pre-Depends, which must be satisfied before a package may be
-	unpacked
-	<item>Depends, which must be satisfied before a package may be
-	configured
-	<item>Recommends, to specify a package which if not installed may
-	severely limit the usefulness of the package
-	<item>Suggests, to specify a package which may increase the
-	productivity of the package
-	<item>Conflicts, to specify a package which must NOT be installed
-	in order for the package to be configured
-	<item>Breaks, to specify a package which is broken by the
-	package and which should therefore not be configured while broken
-	</list>
-Each of these dependencies can specify a version and a depedency on that
-version, for example "<= 0.5-1", "== 2.7.2-1", etc. The comparators available
-are:-
-	<list>
-	<item>"&lt;&lt;" - less than
-	<item>"&lt;=" - less than or equal to
-	<item>"&gt;&gt;" - greater than
-	<item>"&gt;=" - greater than or equal to
-	<item>"==" - equal to
-	</list>
-<item>The concept of "virtual packages", which many other packages may provide,
-using the Provides mechanism. An example of this is the "httpd" virtual package,
-which all web servers should provide. Virtual package names may be used in
-dependency headers. However, current policy is that virtual packages do not
-support version numbers, so dependencies on virtual packages with versions
-will always fail.
-<item>Several other control fields, such as Package, Version, Description,
-Section, Priority, etc., which are mainly for classification purposes. The
-package name must consist entirely of lowercase characters, plus the characters
-'+', '-', and '.'. Fields can extend across multiple lines - on the second
-and subsequent lines, there is a space at the beginning instead of a field
-name and a ':'. Empty lines must consist of the text " .", which will be
-ignored, as will the initial space for other continuation lines. This feature
-is usually only used in the Description field.
-</list>
-
-<sect>The dpkg status area
-
-<p>
-The "dpkg status area" is the term used to refer to the directory where dpkg
-keeps its various status files (GNU would have you call it the dpkg shared
-state directory). This is always, on Debian systems, /var/lib/dpkg. However,
-the default directory name should not be hard-coded, but #define'd, so that
-alteration is possible (it is available via configure in dpkg 1.4.0.9 and
-above). Of course, in a library, code should be allowed to override the
-default directory, but the default should be part of the library (so that
-the user may change the dpkg admin dir simply by replacing the library).
-
-<p>
-Dpkg keeps a variety of files in its status area. These are discussed later
-on in this document, but a quick summary of the files is here:-
-
-<list>
-<item>available - this file contains a concatenation of control information
-from all the packages which dpkg knows about. This is updated using the dpkg
-commands "--update-avail &lt;file&gt;", "--merge-avail &lt;file&gt;", and
-"--clear-avail".
-<item>status - this file contains information on the following things for
-every package:-
-	<list>
-	<item>Whether it is installed, not installed, unpacked, removed,
-		failed configuration, or half-installed (deconfigured in
-		favour of another package).
-	<item>Whether it is selected as install, hold, remove, or purge.
-	<item>If it is "ok" (no installation problems), or "not-ok".
-	<item>It usually also contains the section and priority (so that
-		dselect may classify packages not in available)
-	<item>For packages which did not initially appear in the "available"
-		file when they were installed, the other control information
-		for them.
-	</list>
-   <p>
-   The exact format for the "Status:" field is:
-   <example>
-      Status: Want Flag Status
-   </example>
-   Where <var>Want</> may be one of <em>unknown</>, <em>install</>,
-   <em>hold</>, <em>deinstall</>, <em>purge</>. <var>Flag</>
-   may be one of <em>ok</>, <em>reinstreq</>, <em>hold</>,
-   <em>hold-reinstreq</>.
-   <var>Status</> may be one of <em>not-installed</>, <em>unpacked</>, 
-   <em>half-configured</>, <em>installed</>, <em>half-installed</>
-   <em>config-files</>, <em>post-inst-failed</>, <em>removal-failed</>.
-   The states are as follows:-
-   <taglist>
-     <tag>not-installed
-     <item>No files are installed from the package, it has no config files
-        left, it uninstalled cleanly if it ever was installed.
-     <tag>unpacked
-     <item>The basic files have been unpacked (and are listed in
-        /var/lib/dpkg/info/[package].list. There are config files present,
-        but the postinst script has _NOT_ been run.
-     <tag>half-configured
-     <item>The package was installed and unpacked, but the postinst script
-        failed in some way.
-     <tag>installed
-     <item>All files for the package are installed, and the configuration
-        was also successful.
-     <tag>half-installed
-     <item>An attempt was made to remove the packagem but there was a failure
-        in the prerm script.
-     <tag>config-files
-     <item>The package was "removed", not "purged". The config files are left,
-        but nothing else.
-     <tag>post-inst-failed
-     <item>Old name for half-configured. Do not use.
-     <tag>removal-failed
-     <item>Old name for half-installed. Do not use.
-   </taglist>
-   The two last items are only left in dpkg for compatibility - they are
-   understood by it, but never written out in this form.
-
-   <p>
-   Please see the dpkg source code, <tt>lib/parshelp.c</tt>, 
-   <em>statusinfos</>, <em>eflaginfos</> and <em>wantinfos</> for more 
-   details.
-   
-<item>info - this directory contains files from the control archive of every
-package currently installed. They are installed with a prefix of "&lt;packagename&gt;.".
-In addition to this, it also contains a file called &lt;package&gt;.list for every
-package, which contains a list of files. Note also that the control file is
-not copied into here; it is instead found as part of status or available.
-<item>methods - this directory is reserved for "method"-specific files - each
-"method" has a subdirectory underneath this directory (or at least, it can
-have). In addition, there is another subdirectory "mnt", where misc.
-filesystems (floppies, CD-ROMs, etc.) are mounted.
-<item>alternatives - directory used by the "update-alternatives" program. It
-contains one file for each "alternatives" interface, which contains information
-about all the needed symlinked files for each alternative.
-<item>diversions - file used by the "dpkg-divert" program. Each diversion takes
-three lines. The first is the package name (or ":" for user diversion), the
-second the original filename, and the third the diverted filename.
-<item>updates - directory used internally by dpkg. This is discussed later,
-in the section <ref id="updates">.
-<item>parts - temporary directory used by dpkg-split
-</list>
-
-<sect>The dpkg library files
-
-<p>
-These files are installed under /usr/lib/dpkg (usually), but
-/usr/local/lib/dpkg is also a possibility (as Debian policy dictates). Under
-this directory, there is a "methods" subdirectory. The methods subdirectory
-in turn contains any number of subdirectories for each general method
-processor (note that one set of method scripts can, and is, used for more than
-one of the methods listed under dselect).
-
-<p>
-The following files may be found in each of these subdirectories:-
-
-<list>
-<item>names - One line per method, two-digit priority to appear on menu
-at beginning, followed by a space, the name, and then another space and the
-short description.
-<item>desc.&lt;name&gt; - Contains the long description displayed by dselect
-when the cursor is put over the &lt;name&gt; method.
-<item>setup - Script or program which sets up the initial values to be used
-by this method. Called with first argument as the status area directory
-(/var/lib/dpkg), second argument as the name of the method (as in the directory
-name), and the third argument as the option (as in the names file).
-<item>install - Script/program called when the "install" option of dselect is
-run with this method. Same arguments as for setup.
-<item>update - Script/program called when the "update" option of dselect is
-run. Same arguments as for setup/install.
-</list>
-
-<sect>The "dpkg" command-line utility
-
-<sect1>"Documented" command-line interfaces
-
-<p>
-As yet unwritten. You can refer to the other manuals for now. See
-<manref name="dpkg" section="8">.
-
-<sect1>Environment variables which dpkg responds to
-
-<p>
-<list>
-<item>DPKG_NO_TSTP - if set to a non-null value, this variable causes dpkg to
-run a child shell process instead of sending itself a SIGTSTP, when the user
-selects to background the dpkg process when it asks about conffiles.
-<item>SHELL - used to determine which shell to run in the case when
-DPKG_NO_TSTP is set.
-<item>CC - used as the C compiler to call to determine the target architecture.
-The default is "gcc".
-<item>PATH - dpkg checks that it can find at least the following files in the
-path when it wants to run package installation scripts, and gives an error if
-it cannot find all of them:-
-	<list>
-	<item>ldconfig
-	<item>start-stop-daemon
-	<item>install-info
-	<item>update-rc.d
-	</list>
-</list>
-
-<sect1>Assertions
-
-<p>
-The dpkg utility itself is required for quite a number of packages, even if
-they have been installed with a tool totally separate from dpkg. The reason for
-this is that some packages, in their pre-installation scripts, check that your
-version of dpkg supports certain features. This was broken from the start, and
-it should have actually been a control file header "Dpkg-requires", or similar.
-What happens is that the configuration scripts will abort or continue according
-to the exit code of a call to dpkg, which will stop them from being wrongly
-configured.
-
-<p>
-These special command-line options, which simply return as true or false are
-all prefixed with "--assert-". Here is a list of them (without the prefix):-
-
-<list>
-<item>support-predepends - Returns success or failure according to whether
-a version of dpkg which supports predepends properly (1.1.0 or above) is
-installed, according to the database.
-<item>working-epoch - Return success or failure according to whether a version
-of dpkg which supports epochs in version properly (1.4.0.7 or above) is
-installed, according to the database.
-</list>
-
-<p>
-Both these options check the status database to see what version of the "dpkg"
-package is installed, and check it against a known working version.
-
-<sect1>--predep-package
-
-<p>
-This strange option is described as follows in the source code:
-
-<example>
-/* Print a single package which:
- *  (a) is the target of one or more relevant predependencies.
- *  (b) has itself no unsatisfied pre-dependencies.
- * If such a package is present output is the Packages file entry,
- *  which can be massaged as appropriate.
- * Exit status:
- *  0 = a package printed, OK
- *  1 = no suitable package available
- *  2 = error
- */
-</example>
-
-<p>
-On further inspection of the source code, it appears that what is does is
-this:-
-
-<list>
-<item>Looks at the packages in the database which are selected as "install",
-and are installed.
-<item>It then looks at the Pre-Depends information for each of these packages
-from the available file. When it find a package for which any of the
-pre-dependencies are not satisfied, it breaks from the loop through the packages.
-<item>It then looks through the unsatisfied pre-dependencies, and looks for
-packages which would satisfy this pre-dependency, stopping on the first it
-finds. If it finds none, it bombs out with an error.
-<item>It then continues this for every dependency of the initial package.
-</list>
-
-Eventually, it writes out the record of all the packages to satisfy the
-pre-dependencies. This is used by the disk method to make sure that its
-dependency ordering is correct. What happens is that all pre-depending
-packages are first installed, then it runs dpkg -iGROEB on the directory,
-which installs in the order package files are found. Since pre-dependencies
-mean that a package may not even be unpacked unless they are satisfied, it is
-necessary to do this (usually, since all the package files are unpacked in one
-phase, the configured in another, this is not needed).
-
-<chapt>dpkg-deb and .deb file internals
-
-<p>
-This chapter describes the internals to the "dpkg-deb" tool, which is used
-by "dpkg" as a back-end. dpkg-deb has its own tar extraction functions, which
-is the source of many problems, as it does not support long filenames, using
-extension blocks.
-
-<sect>The .deb archive format
-
-<p>
-The main principal of the new-format Debian archive (I won't describe the old
-format - for that have a look at deb-old.5), is that the archive really is
-an archive - as used by "ar" and friends. However, dpkg-deb uses this format
-internally, rather than calling "ar". Inside this archive, there are usually
-the following members:-
-
-<list>
-<item>debian-binary
-<item>control.tar.gz
-<item>data.tar.gz
-</list>
-
-<p>
-The debian-binary member consists simply of the string "2.0", indicating the
-format version. control.tar.gz contains the control files (and scripts), and
-the data.tar.gz contains the actual files to populate the filesystem with.
-Both tarfiles extract straight into the current directory. Information on the
-tar formats can be found in the GNU tar info page. Since dpkg-deb calls
-"tar -cf" to build packages, the Debian packages use the GNU extensions.
-
-<sect>The dpkg-deb command-line 
-
-<p>
-dpkg-deb documents itself thoroughly with its '--help' command-line option.
-However, I am including a reference to these for completeness. dpkg-deb
-supports the following options:-
-
-<list>
-<item>--build (-b) &lt;dir&gt; - builds a .deb archive, takes a directory which
-contains all the files as an argument. Note that the directory
-&lt;dir&gt;/DEBIAN will be packed separately into the control archive.
-<item>--contents (-c) &lt;debfile&gt; - Lists the contents of the "data.tar.gz"
-member.
-<item>--control (-e) &lt;debfile&gt; - Extracts the control archive into a
-directory called DEBIAN. Alternatively, with another argument, it will extract
-it into a different directory.
-<item>--info (-I) &lt;debfile&gt; - Prints the contents of the "control" file
-in the control archive to stdout. Alternatively, giving it other arguments will
-cause it to print the contents of those files instead.
-<item>--field (-f) &lt;debfile&gt; &lt;field&gt; ... - Prints any number of
-fields from the "control" file. Giving it extra arguments limits the fields it
-prints to only those specified. With no command-line arguments other than a
-filename, it is equivalent to -I and just the .deb filename.
-<item>--extract (-x) &lt;debfile&gt; &lt;dir&gt; - Extracts the data archive
-of a debian package under the directory &lt;dir&gt;.
-<item>--vextract (-X) &lt;debfile&gt; &lt;dir&gt; - Same as --extract, except
-it is equivalent of giving tar the '-v' option - it prints the filenames as
-it extracts them.
-<item>--fsys-tarfile &lt;debfile&gt; - This option outputs a gunzip'd version
-of data.tar.gz to stdout.
-<item>--new - sets the archive format to be used to the new Debian format
-<item>--old - sets the archive format to be used to the old Debian format
-<item>--debug - Tells dpkg-deb to produce debugging output
-<item>--nocheck - Tells dpkg-deb not to check the sanity of the control file
-<item>--help (-h) - Gives a help message
-<item>--version - Shows the version number
-<item>--licence/--license (UK/US spellings) - Shows a brief outline of the GPL
-</list>
-
-<sect1>Internal checks used by dpkg-deb when building packages
-
-<p>
-Here is a list of the internal checks used by dpkg-deb when building packages.
-It is in the order they are done.
-
-<list>
-<item>First, the output Debian archive argument, if it is given, is checked
-using stat. If it is a directory, an internal flag is set. This check is only
-made if the archive name is specified explicitly on the command-line. If the
-argument was not given, the default is the directory name, with ".deb"
-appended.
-<item>Next, the control file is checked, unless the --nocheck flag was
-specified on the command-line. dpkg-deb will bomb out if the second argument
-to --build was a directory, and --nocheck was specified. Note that dpkg-deb
-will not be able to determine the name of the package in this case. In the
-control file, the following things are checked:-
-	<list>
-	<item>The package name is checked to see if it contains any invalid
-	characters (see <ref id="control"> for this).
-	<item>The priority field is checked to see if it uses standard values,
-	and user-defined values are warned against. However, note that this
-	check is now redundant, since the control file no longer contains
-	the priority - the changes file now does this.
-	<item>The control file fields are then checked against the standard
-	list of fields which appear in control files, and any "user-defined"
-	fields are reported as warnings.
-	<item>dpkg-deb then checks that the control file contains a valid
-	version number.
-	</list>
-<item>After this, in the case where a directory was specified to build the
-.deb file in, the filename is created as "directory/pkg_ver.deb" or
-"directory/pkg_ver_arch.deb", depending on whether the control file contains
-an architecture field.
-<item>Next, dpkg-deb checks for the &lt;dir&gt;/DEBIAN directory. It complains
-if it doesn't exist, or if it has permissions &lt; 0755, or &gt; 0775.
-<item>It then checks that all the files in this subdir are either symlinks
-or plain files, and have permissions between 0555 and 0775.
-<item>The conffiles file is then checked to see if the filenames are too
-long. Warnings are produced for each that is. After this, it checks that
-the package provides initial copies of each of these conffiles, and that
-they are all plain files.
-</list>
-
-<chapt>dpkg internals
-
-<p>
-This chapter describes the internals of dpkg itself. Although the low-level
-formats are quite simple, what dpkg does in certain cases often does not
-make sense.
-
-<sect id="updates">Updates
-
-<p>
-This describes the /var/lib/dpkg/updates directory. The function of this
-directory is somewhat strange, and seems only to be used internally. A function
-called cleanupdates is called whenever the database is scanned. This function
-in turn uses <manref name="scandir" section="3">, to sort the files in this
-directory. Files who names do not consist entirely of digits are discarded.
-dpkg also causes a fatal error if any of the filenames are different lengths.
-
-<p>
-After having scanned the directory, dpkg in turn parses each file the same way
-it parses the status file (they are sorted by the scandir to be in numerical
-order). After having done this, it then writes the status information back
-to the "status" file, and removes all the "updates" files.
-
-<p>
-These files are created internally by dpkg's "checkpoint" function, and are
-cleaned up when dpkg exits cleanly.
-
-<p>
-Juding by the use of the updates directory I would call it a Journal. Inorder
-to efficiently ensure the complete integrity of the status file dpkg will
-"checkpoint" or journal all of it's activities in the updates directory. By
-merging the contents of the updates directory (in order!!) against the 
-original status file it can get the precise current state of the system,
-even in the event of a system failure while dpkg is running.
-
-<p> 
-The other option would be to sync-rewrite the status file after each 
-operation, which would kill performance.
-
-<p>
-It is very important that any program that uses the status file abort if
-the updates directory is not empty! The user should be informed to run dpkg
-manually (what options though??) to correct the situation.
-
-<sect>What happens when dpkg reads the database
-
-<p>
-First, the status file is read. This gives dpkg an initial idea of the packages
-that are there. Next, the updates files are read in, overriding the status
-file, and if necessary, the status file is re-written, and updates files are
-removed. Finally, the available file is read. The available file is read
-with flags which preclude dpkg from updating any status information from it,
-though - installed version, etc., and is also told to record that the packages
-it reads this time are available, not installed.
-
-<p>
-More information on updates is given above.
-
-<sect>How dpkg compares version numbers
-
-<p>
-Version numbers consist of three parts: the epoch, the upstream version, and
-the Debian revision. Dpkg compares these parts in that order. If the epochs
-are different, it returns immediately, and so on.
-
-<p>
-However, the important part is how it compares the versions which are
-essentially stored as just strings. These are compared in two distinct parts:
-those consisting of numerical characters (which are evaluated, and then
-compared), and those consisting of other characters. When comparing
-non-numerical parts, they are compared as the character values (ASCII), but
-non-alphabetical characters are considered "greater than" alphabetical ones.
-Also note that longer strings (after excluding differences where numerical
-values are equal) are considered "greater than" shorter ones.
-
-<p>
-Here are a few examples of how these rules apply:-
-
-<example>
-15 > 10
-0010 == 10
-
-d.r > dsr
-32.d.r == 0032.d.r
-d.rnr < d.rnrn
-</example>
-
-</book>

doc/examples/configure-index

@@ -414,8 +414,8 @@ DPkg
    FlushSTDIN "true";
 
    // Control the size of the command line passed to dpkg.
-   MaxBytes 1024;
-   MaxArgs 350;
+   MaxArgBytes 32768;
+   MaxArgs 8192;
 
    // controls if apt will apport on the first dpkg error or if it 
    // tries to install as many packages as possible

doc/files.dbk

@@ -0,0 +1,393 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
+<title>APT Files</title>
+
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Jason Gunthorpe</personname><email>jgg@debian.org</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
+
+<abstract>
+<para>
+This document describes the complete implementation and format of the installed
+APT directory structure. It also serves as guide to how APT views the Debian
+archive.
+</para>
+</abstract>
+
+<copyright><year>1998-1999</year><holder>Jason Gunthorpe</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
+"APT" and this document are free software; you can redistribute them and/or
+modify them under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at your
+option) any later version.
+</para>
+<para>
+For more details, on Debian systems, see the file
+/usr/share/common-licenses/GPL for the full license.
+</para>
+</legalnotice>
+
+</bookinfo>
+
+<chapter id="ch1"><title>Introduction</title>
+
+<section id="s1.1"><title>General</title>
+<para>
+This document serves two purposes. The first is to document the installed
+directory structure and the format and purpose of each file. The second
+purpose is to document how APT views the Debian archive and deals with multiple
+package files.
+</para>
+<para>
+The var directory structure is as follows:
+</para>
+<screen>
+  /var/lib/apt/
+		lists/
+		       partial/
+		periodic/
+		extended_states
+		cdroms.list
+  /var/cache/apt/
+		  archives/
+		          partial/
+		  pkgcache.bin
+		  srcpkgcache.bin
+  /etc/apt/
+	    sources.list.d/
+	    apt.conf.d/
+	    preferences.d/
+	    trusted.gpg.d/
+	    sources.list
+	    apt.conf
+	    apt_preferences
+	    trusted.gpg
+  /usr/lib/apt/
+	        methods/
+			 bzip2
+			 cdrom
+			 copy
+			 file
+			 ftp
+			 gpgv
+			 gzip
+			 http
+			 https
+			 lzma
+			 rred
+			 rsh
+			 ssh
+</screen>
+<para>
+As is specified in the FHS 2.1 /var/lib/apt is used for application data that
+is not expected to be user modified. /var/cache/apt is used for regeneratable
+data and is where the package cache and downloaded .debs go. /etc/apt is the
+place where configuration should happen and /usr/lib/apt is the place where the
+apt and other packages can place binaries which can be used by the acquire
+system of APT.
+</para>
+</section>
+
+</chapter>
+
+<chapter id="ch2"><title>Files</title>
+
+<section id="s2.1"><title>Files and fragment directories in /etc/apt</title>
+<para>
+All files in /etc/apt are used to modify specific aspects of APT. To enable
+other packages to ship needed configuration herself all these files have a
+fragment directory packages can place their files in instead of mangling with
+the main files. The main files are therefore considered to be only used by the
+user and not by a package. The documentation omits this directories most of
+the time to be easier readable, so every time the documentation includes a
+reference to a main file it really means the file or the fragment directories.
+</para>
+</section>
+
+<section id="s2.2"><title>Distribution Source list (sources.list)</title>
+<para>
+The distribution source list is used to locate archives of the debian
+distribution. It is designed to support any number of active sources and to
+support a mix of source media. The file lists one source per line, with the
+fastest source listed first. The format of each line is:
+</para>
+<para>
+<replaceable>type uri args</replaceable>
+</para>
+<para>
+The first item, <replaceable>type</replaceable>, indicates the format for the
+remainder of the line. It is designed to indicate the structure of the
+distribution the line is talking about. Currently the only defined values are
+<emphasis>deb</emphasis> and <emphasis>deb-src</emphasis> which indicate a
+standard debian (source) archive with a dists directory. More about these
+types and the URI specification can be found in the sources.list manpage.
+</para>
+
+<section id="s2.2.1"><title>Hashing the URI</title>
+<para>
+All permanent information acquired from any of the sources is stored in the
+lists directory. Thus, there must be a way to relate the filename in the lists
+directory to a line in the sourcelist. To simplify things this is done by
+quoting the URI and treating _'s as quoteable characters and converting /
+to _. The URI spec says this is done by converting a sensitive character
+into %xx where xx is the hexadecimal representation from the ASCII character
+set. Examples:
+</para>
+<screen>
+http://www.debian.org/archive/dists/stable/binary-i386/Packages
+/var/lib/apt/lists/www.debian.org_archive_dists_stable_binary-i386_Packages
+
+cdrom:Debian 1.3/debian/Packages
+/var/lib/apt/info/Debian%201.3_debian_Packages
+</screen>
+<para>
+The other alternative that was considered was to use a deep directory structure
+but this poses two problems, it makes it very difficult to prune directories
+back when sources are no longer used and complicates the handling of the
+partial directory. This gives a very simple way to deal with all of the
+situations that can arise. Also note that the same rules described in the
+<emphasis>Archive Directory</emphasis> section regarding the partial sub dir
+apply here as well.
+</para>
+</section>
+
+</section>
+
+<section id="s2.3"><title>Extended States File (extended_states)</title>
+<para>
+The extended_states file serves the same purpose as the normal dpkg status
+file (/var/lib/dpkg/status) except that it stores information unique to
+apt. This includes currently only the autoflag but is open to store more
+unique data that come up over time. It duplicates nothing from the normal
+dpkg status file. Please see other APT documentation for a discussion of
+the exact internal behavior of these fields. The Package and the Architecture
+field are placed directly before the new fields to indicate which package
+they apply to. The new fields are as follows:
+</para>
+<variablelist>
+<varlistentry>
+<term>Auto-Installed</term>
+<listitem>
+<para>
+The Auto flag can be 1 (Yes) or 0 (No) and controls whether the package was
+automatical installed to satisfy a dependency or if the user requested the
+installation
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</section>
+
+<section id="s2.4"><title>Binary Package Cache (srcpkgcache.bin and pkgcache.bin)</title>
+<para>
+Please see cache.sgml for a complete description of what this file
+is. The cache file is updated whenever the contents of the lists
+directory changes. If the cache is erased, corrupted or of a non-matching
+version it will be automatically rebuilt by all of the tools that need
+it. <emphasis>srcpkgcache.bin</emphasis> contains a cache of all of the
+package files in the source list. This allows regeneration of the cache
+when the status files change to use a prebuilt version for greater speed.
+</para>
+</section>
+
+<section id="s2.5"><title>Downloads Directory (archives)</title>
+<para>
+The archives directory is where all downloaded .deb archives go. When the file
+transfer is initiated the deb is placed in partial. Once the file is fully
+downloaded and its MD5 hash and size are verified it is moved from partial
+into archives/. Any files found in archives/ can be assumed to be verified.
+</para>
+<para>
+No directory structure is transferred from the receiving site and all .deb file
+names conform to debian conventions. No short (msdos) filename should be
+placed in archives. If the need arises .debs should be unpacked, scanned and
+renamed to their correct internal names. This is mostly to prevent file name
+conflicts but other programs may depend on this if convenient. A conforming
+.deb is one of the form, name_version_arch.deb. Our archive scripts do not
+handle epochs, but they are necessary and should be re-inserted. If necessary
+_'s and :'s in the fields should be quoted using the % convention. It must be
+possible to extract all 3 fields by examining the file name. Downloaded .debs
+must be found in one of the package lists with an exact name + version match..
+</para>
+</section>
+
+<section id="s2.6"><title>The Methods Directory (/usr/lib/apt/methods)</title>
+<para>
+The Methods directory is more fully described in the APT Methods interface
+document.
+</para>
+</section>
+
+<section id="s2.7"><title>The Configuration File (/etc/apt/apt.conf)</title>
+<para>
+The configuration file (and the associated fragments directory
+/etc/apt/apt.conf.d/) is described in the apt.conf manpage.
+</para>
+</section>
+
+<section id="s2.8"><title>The trusted.gpg File (/etc/apt/trusted.gpg)</title>
+<para>
+The trusted.gpg file (and the files in the associated fragments directory
+/etc/apt/trusted.gpg.d/) is a binary file including the keyring used by apt to
+validate that the information (e.g. the Release file) it downloads are really
+from the distributor it clams to be and is unmodified and is therefore the last
+step in the chain of trust between the archive and the end user. This security
+system is described in the apt-secure manpage.
+</para>
+</section>
+
+<section id="s2.9"><title>The Release File</title>
+<para>
+This file plays an important role in how APT presents the archive to the
+user. Its main purpose is to present a descriptive name for the source of
+each version of each package. It also is used to detect when new versions
+of debian are released. It augments the package file it is associated with
+by providing meta information about the entire archive which the Packages
+file describes.
+</para>
+<para>
+The full name of the distribution for presentation to the user is formed as
+'label version archive', with a possible extended name being 'label version
+archive component'.
+</para>
+<para>
+The file is formed as the package file (RFC-822) with the following tags
+defined:
+</para>
+<variablelist>
+<varlistentry>
+<term>Archive</term>
+<listitem>
+<para>
+This is the common name we give our archives, such as
+<emphasis>stable</emphasis> or <emphasis>unstable</emphasis>.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Component</term>
+<listitem>
+<para>
+Refers to the sub-component of the archive, <emphasis>main</emphasis>,
+<emphasis>contrib</emphasis> etc. Component may be omitted if there are no
+components for this archive.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Version</term>
+<listitem>
+<para>
+This is a version string with the same properties as in the Packages file. It
+represents the release level of the archive.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Origin</term>
+<listitem>
+<para>
+This specifies who is providing this archive. In the case of Debian the string
+will read 'Debian'. Other providers may use their own string
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Label</term>
+<listitem>
+<para>
+This carries the encompassing name of the distribution. For Debian proper this
+field reads 'Debian'. For derived distributions it should contain their proper
+name.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Architecture</term>
+<listitem>
+<para>
+When the archive has packages for a single architecture then the Architecture
+is listed here. If a mixed set of systems are represented then this should
+contain the keyword <emphasis>mixed</emphasis>.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>NotAutomatic</term>
+<listitem>
+<para>
+A Yes/No flag indicating that the archive is extremely unstable and its
+version's should never be automatically selected. This is to be used by
+experimental.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Description</term>
+<listitem>
+<para>
+Description is used to describe the release. For instance experimental would
+contain a warning that the packages have problems.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+<para>
+The location of the Release file in the archive is very important, it must be
+located in the same location as the packages file so that it can be located in
+all situations. The following is an example for the current stable release,
+1.3.1r6
+</para>
+<screen>
+Archive: stable
+Component: main
+Version: 1.3.1r6
+Origin: Debian
+Label: Debian
+Architecture: i386
+</screen>
+<para>
+This is an example of experimental,
+</para>
+<screen>
+Archive: experimental
+Version: 0
+Origin: Debian
+Label: Debian
+Architecture: mixed
+NotAutomatic: Yes
+</screen>
+<para>
+And unstable,
+</para>
+<screen>
+Archive: unstable
+Component: main
+Version: 2.1
+Origin: Debian
+Label: Debian
+Architecture: i386
+</screen>
+</section>
+
+</chapter>
+
+
+</book>

doc/files.sgml

@@ -1,345 +0,0 @@
-<!-- -*- mode: sgml; mode: fold -*- -->
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<book>
-<title>APT Files</title>
-
-<author>Jason Gunthorpe <email>jgg@debian.org</email></author>
-<version>$Id: files.sgml,v 1.12 2003/04/26 23:26:13 doogie Exp $</version>
-
-<abstract>
-This document describes the complete implementation and format of the 
-installed APT directory structure. It also serves as guide to how APT 
-views the Debian archive.
-</abstract>
-
-<copyright>
-Copyright &copy; Jason Gunthorpe, 1998-1999.
-<p>
-"APT" and this document are free software; you can redistribute them and/or
-modify them under the terms of the GNU General Public License as published
-by the Free Software Foundation; either version 2 of the License, or (at your
-option) any later version.
-
-<p>
-For more details, on Debian systems, see the file
-/usr/share/common-licenses/GPL for the full license.
-</copyright>
-
-<toc sect>
-
-<chapt>Introduction
-<!-- General		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>General
-
-<p>
-This document serves two purposes. The first is to document the installed
-directory structure and the format and purpose of each file. The second
-purpose is to document how APT views the Debian archive and deals with 
-multiple package files.
-
-<p>
-The var directory structure is as follows:
-<example>
-  /var/lib/apt/
-		lists/
-		       partial/
-		periodic/
-		extended_states
-		cdroms.list
-  /var/cache/apt/
-		  archives/
-		          partial/
-		  pkgcache.bin
-		  srcpkgcache.bin
-  /etc/apt/
-	    sources.list.d/
-	    apt.conf.d/
-	    preferences.d/
-	    trusted.gpg.d/
-	    sources.list
-	    apt.conf
-	    apt_preferences
-	    trusted.gpg
-  /usr/lib/apt/
-	        methods/
-			 bzip2
-			 cdrom
-			 copy
-			 file
-			 ftp
-			 gpgv
-			 gzip
-			 http
-			 https
-			 lzma
-			 rred
-			 rsh
-			 ssh
-</example>
-
-<p>
-As is specified in the FHS 2.1 /var/lib/apt is used for application 
-data that is not expected to be user modified. /var/cache/apt is used
-for regeneratable data and is where the package cache and downloaded .debs
-go. /etc/apt is the place where configuration should happen and
-/usr/lib/apt is the place where the apt and other packages can place
-binaries which can be used by the acquire system of APT.
-</sect>
-                                                                  <!-- }}} -->
-
-<chapt>Files
-<!-- Distribution Source List					       {{{ -->
-<!-- ===================================================================== -->
-<sect>Files and fragment directories in /etc/apt
-
-<p>
-All files in /etc/apt are used to modify specific aspects of APT. To enable
-other packages to ship needed configuration herself all these files have
-a fragment directory packages can place their files in instead of mangling
-with the main files. The main files are therefore considered to be only
-used by the user and not by a package. The documentation omits this directories
-most of the time to be easier readable, so every time the documentation includes
-a reference to a main file it really means the file or the fragment directories.
-
-</sect>
-
-<sect>Distribution Source list (sources.list)
-
-<p>
-The distribution source list is used to locate archives of the debian
-distribution. It is designed to support any number of active sources and to
-support a mix of source media. The file lists one source per line, with the 
-fastest source listed first. The format of each line is:
-
-<p>
-<var>type uri args</var>
-
-<p>
-The first item, <var>type</var>, indicates the format for the remainder 
-of the line. It is designed to indicate the structure of the distribution
-the line is talking about. Currently the only defined values are <em>deb</em>
-and <em>deb-src</em> which indicate a standard debian (source) archive with a
-dists directory. More about these types and the URI specification can be found
-in the sources.list manpage.
-
-<sect1>Hashing the URI
-<p>
-All permanent information acquired from any of the sources is stored in the
-lists directory. Thus, there must be a way to relate the filename in the
-lists directory to a line in the sourcelist. To simplify things this is
-done by quoting the URI and treating _'s as quoteable characters and
-converting / to _. The URI spec says this is done by converting a 
-sensitive character into %xx where xx is the hexadecimal representation 
-from the ASCII character set. Examples:
-
-<example>
-http://www.debian.org/archive/dists/stable/binary-i386/Packages 
-/var/lib/apt/lists/www.debian.org_archive_dists_stable_binary-i386_Packages
-
-cdrom:Debian 1.3/debian/Packages
-/var/lib/apt/info/Debian%201.3_debian_Packages
-</example>
-
-<p> 
-The other alternative that was considered was to use a deep directory 
-structure but this poses two problems, it makes it very difficult to prune
-directories back when sources are no longer used and complicates the handling
-of the partial directory. This gives a very simple way to deal with all
-of the situations that can arise. Also note that the same rules described in 
-the <em>Archive Directory</> section regarding the partial sub dir apply 
-here as well.
-</sect1>
-
-</sect>
-                                                                  <!-- }}} -->
-<!-- Extended Status						       {{{ -->
-<!-- ===================================================================== -->
-<sect>Extended States File (extended_states)
-
-<p>
-The extended_states file serves the same purpose as the normal dpkg status file
-(/var/lib/dpkg/status) except that it stores information unique to apt.
-This includes currently only the autoflag but is open to store more
-unique data that come up over time. It duplicates nothing from the normal
-dpkg status file.  Please see other APT documentation for a discussion
-of the exact internal behavior of these fields. The Package and the
-Architecture field are placed directly before the new fields to indicate
-which package they apply to. The new fields are as follows:
-
-<taglist>
-<tag>Auto-Installed<item>
-   The Auto flag can be 1 (Yes) or 0 (No) and controls whether the package
-   was automatical installed to satisfy a dependency or if the user requested
-   the installation
-</taglist>
-</sect>
-                                                                  <!-- }}} -->
-<!-- Binary Package Cache					       {{{ -->
-<!-- ===================================================================== -->
-<sect>Binary Package Cache (srcpkgcache.bin and pkgcache.bin)
-
-<p>
-Please see cache.sgml for a complete description of what this file is. The 
-cache file is updated whenever the contents of the lists directory changes.
-If the cache is erased, corrupted or of a non-matching version it will
-be automatically rebuilt by all of the tools that need it. 
-<em>srcpkgcache.bin</> contains a cache of all of the package files in the 
-source list. This allows regeneration of the cache when the status files 
-change to use a prebuilt version for greater speed.
-</sect>
-                                                                  <!-- }}} -->
-<!-- Downloads Directory					       {{{ -->
-<!-- ===================================================================== -->
-<sect>Downloads Directory (archives)
-
-<p>
-The archives directory is where all downloaded .deb archives go. When the
-file transfer is initiated the deb is placed in partial. Once the file
-is fully downloaded and its MD5 hash and size are verified it is moved
-from partial into archives/. Any files found in archives/ can be assumed 
-to be verified.
-
-<p>
-No directory structure is transferred from the receiving site and all .deb
-file names conform to debian conventions. No short (msdos) filename should
-be placed in archives. If the need arises .debs should be unpacked, scanned
-and renamed to their correct internal names. This is mostly to prevent
-file name conflicts but other programs may depend on this if convenient. 
-A conforming .deb is one of the form, name_version_arch.deb. Our archive
-scripts do not handle epochs, but they are necessary and should be re-inserted.
-If necessary _'s and :'s in the fields should be quoted using the % convention.
-It must be possible to extract all 3 fields by examining the file name.
-Downloaded .debs must be found in one of the package lists with an exact
-name + version match..
-</sect>
-                                                                  <!-- }}} -->
-<!-- The Methods Directory					       {{{ -->
-<!-- ===================================================================== -->
-<sect> The Methods Directory (/usr/lib/apt/methods)
-
-<p>
-The Methods directory is more fully described in the APT Methods interface
-document.
-</sect>
-                                                                  <!-- }}} -->
-<!-- The Configuration File					       {{{ -->
-<!-- ===================================================================== -->
-<sect> The Configuration File (/etc/apt/apt.conf)
-
-<p>
-The configuration file (and the associated fragments directory
-/etc/apt/apt.conf.d/) is described in the apt.conf manpage.
-</sect>
-                                                                  <!-- }}} -->
-<!-- The trusted.gpg File					       {{{ -->
-<!-- ===================================================================== -->
-<sect> The trusted.gpg File (/etc/apt/trusted.gpg)
-
-<p>
-The trusted.gpg file (and the files in the associated fragments directory
-/etc/apt/trusted.gpg.d/) is a binary file including the keyring used
-by apt to validate that the information (e.g. the Release file) it
-downloads are really from the distributor it clams to be and is
-unmodified and is therefore the last step in the chain of trust between
-the archive and the end user. This security system is described in the
-apt-secure manpage.
-</sect>
-                                                                  <!-- }}} -->
-<!-- The Release File						       {{{ -->
-<!-- ===================================================================== -->
-<sect> The Release File
-
-<p>
-This file plays an important role in how APT presents the archive to the 
-user. Its main purpose is to present a descriptive name for the source
-of each version of each package. It also is used to detect when new versions
-of debian are released. It augments the package file it is associated with 
-by providing meta information about the entire archive which the Packages
-file describes.
-
-<p>
-The full name of the distribution for presentation to the user is formed
-as 'label version archive', with a possible extended name being 
-'label version archive component'.
-
-<p>
-The file is formed as the package file (RFC-822) with the following tags
-defined:
-
-<taglist>
-<tag>Archive<item>
-This is the common name we give our archives, such as <em>stable</> or
-<em>unstable</>.
-
-<tag>Component<item>
-Refers to the sub-component of the archive, <em>main</>, <em>contrib</>
-etc. Component may be omitted if there are no components for this archive.
-
-<tag>Version<item>
-This is a version string with the same properties as in the Packages file.
-It represents the release level of the archive.
-
-<tag>Origin<item>
-This specifies who is providing this archive. In the case of Debian the
-string will read 'Debian'. Other providers may use their own string
-
-<tag>Label<item>
-This carries the encompassing name of the distribution. For Debian proper
-this field reads 'Debian'. For derived distributions it should contain their 
-proper name.
-
-<tag>Architecture<item>
-When the archive has packages for a single architecture then the Architecture
-is listed here. If a mixed set of systems are represented then this should
-contain the keyword <em>mixed</em>.
-
-<tag>NotAutomatic<item>
-A Yes/No flag indicating that the archive is extremely unstable and its
-version's should never be automatically selected. This is to be used by 
-experimental.
-
-<tag>Description<item>
-Description is used to describe the release. For instance experimental would
-contain a warning that the packages have problems.
-</taglist>
-
-<p>
-The location of the Release file in the archive is very important, it must 
-be located in the same location as the packages file so that it can be 
-located in all situations. The following is an example for the current stable
-release, 1.3.1r6 
-
-<example>
-Archive: stable
-Component: main
-Version: 1.3.1r6
-Origin: Debian
-Label: Debian
-Architecture: i386
-</example>
-
-This is an example of experimental,
-<example>
-Archive: experimental
-Version: 0
-Origin: Debian
-Label: Debian
-Architecture: mixed
-NotAutomatic: Yes
-</example>
-
-And unstable,
-<example>
-Archive: unstable
-Component: main
-Version: 2.1
-Origin: Debian
-Label: Debian
-Architecture: i386
-</example>
-
-</sect>
-                                                                  <!-- }}} -->
-
-</book>

doc/guide.dbk

@@ -0,0 +1,561 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
+<title>APT User's Guide</title>
+
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Jason Gunthorpe</personname><email>jgg@debian.org</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
+
+<abstract>
+<para>
+This document provides an overview of how to use the the APT package manager.
+</para>
+</abstract>
+
+<copyright><year>1998</year><holder>Jason Gunthorpe</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
+"APT" and this document are free software; you can redistribute them and/or
+modify them under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at your
+option) any later version.
+</para>
+</legalnotice>
+
+<legalnotice>
+<para>
+For more details, on Debian systems, see the file
+/usr/share/common-licenses/GPL for the full license.
+</para>
+</legalnotice>
+
+</bookinfo>
+
+<chapter id="ch1"><title>General</title>
+<para>
+The APT package currently contains two sections, the APT
+<command>dselect</command> method and the <command>apt-get</command> command
+line user interface. Both provide a way to install and remove packages as well
+as download new packages from the Internet.
+</para>
+
+<section id="s1.1"><title>Anatomy of the Package System</title>
+<para>
+The Debian packaging system has a large amount of information associated with
+each package to help assure that it integrates cleanly and easily into the
+system. The most prominent of its features is the dependency system.
+</para>
+<para>
+The dependency system allows individual programs to make use of shared elements
+in the system such as libraries. It simplifies placing infrequently used
+portions of a program in separate packages to reduce the number of things the
+average user is required to install. Also, it allows for choices in mail
+transport agents, X servers and so on.
+</para>
+<para>
+The first step to understanding the dependency system is to grasp the concept
+of a simple dependency. The meaning of a simple dependency is that a package
+requires another package to be installed at the same time to work properly.
+</para>
+<para>
+For instance, mailcrypt is an emacs extension that aids in encrypting email
+with GPG. Without GPGP installed mailcrypt is useless, so mailcrypt has a
+simple dependency on GPG. Also, because it is an emacs extension it has a
+simple dependency on emacs, without emacs it is completely useless.
+</para>
+<para>
+The other important dependency to understand is a conflicting dependency. It
+means that a package, when installed with another package, will not work and
+may possibly be extremely harmful to the system. As an example consider a mail
+transport agent such as sendmail, exim or qmail. It is not possible to have
+two mail transport agents installed because both need to listen to the network
+to receive mail. Attempting to install two will seriously damage the system so
+all mail transport agents have a conflicting dependency with all other mail
+transport agents.
+</para>
+<para>
+As an added complication there is the possibility for a package to pretend to
+be another package. Consider that exim and sendmail for many intents are
+identical, they both deliver mail and understand a common interface. Hence,
+the package system has a way for them to declare that they are both
+mail-transport-agents. So, exim and sendmail both declare that they provide a
+mail-transport-agent and other packages that need a mail transport agent depend
+on mail-transport-agent. This can add a great deal of confusion when trying to
+manually fix packages.
+</para>
+<para>
+At any given time a single dependency may be met by packages that are already
+installed or it may not be. APT attempts to help resolve dependency issues by
+providing a number of automatic algorithms that help in selecting packages for
+installation.
+</para>
+</section>
+
+</chapter>
+
+<chapter id="ch2"><title>apt-get</title>
+<para>
+<command>apt-get</command> provides a simple way to install packages from the
+command line. Unlike <command>dpkg</command>, <command>apt-get</command> does
+not understand .deb files, it works with the package's proper name and can only
+install .deb archives from a <emphasis>Source</emphasis>.
+</para>
+<para>
+The first <footnote><para> If you are using an http proxy server you must set
+the http_proxy environment variable first, see sources.list(5) </para>
+</footnote> thing that should be done before using <command>apt-get</command>
+is to fetch the package lists from the <emphasis>Sources</emphasis> so that it
+knows what packages are available. This is done with <literal>apt-get
+update</literal>. For instance,
+</para>
+<screen>
+# apt-get update
+Get http://ftp.de.debian.org/debian-non-US/ stable/binary-i386/ Packages
+Get http://llug.sep.bnl.gov/debian/ testing/contrib Packages
+Reading Package Lists... Done
+Building Dependency Tree... Done
+</screen>
+<para>
+Once updated there are several commands that can be used:
+</para>
+<variablelist>
+<varlistentry>
+<term>upgrade</term>
+<listitem>
+<para>
+Upgrade will attempt to gently upgrade the whole system. Upgrade will never
+install a new package or remove an existing package, nor will it ever upgrade a
+package that might cause some other package to break. This can be used daily
+to relatively safely upgrade the system. Upgrade will list all of the packages
+that it could not upgrade, this usually means that they depend on new packages
+or conflict with some other package. <command>dselect</command> or
+<literal>apt-get install</literal> can be used to force these packages to
+install.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>install</term>
+<listitem>
+<para>
+Install is used to install packages by name. The package is automatically
+fetched and installed. This can be useful if you already know the name of the
+package to install and do not want to go into a GUI to select it. Any number
+of packages may be passed to install, they will all be fetched. Install
+automatically attempts to resolve dependency problems with the listed packages
+and will print a summary and ask for confirmation if anything other than its
+arguments are changed.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>dist-upgrade</term>
+<listitem>
+<para>
+Dist-upgrade is a complete upgrader designed to simplify upgrading between
+releases of Debian. It uses a sophisticated algorithm to determine the best
+set of packages to install, upgrade and remove to get as much of the system to
+the newest release. In some situations it may be desired to use dist-upgrade
+rather than spend the time manually resolving dependencies in
+<command>dselect</command>. Once dist-upgrade has completed then
+<command>dselect</command> can be used to install any packages that may have
+been left out.
+</para>
+<para>
+It is important to closely look at what dist-upgrade is going to do, its
+decisions may sometimes be quite surprising.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+<para>
+<command>apt-get</command> has several command line options that are detailed
+in its man page,
+<citerefentry><refentrytitle>apt-get</refentrytitle><manvolnum>8</manvolnum></citerefentry>. The
+most useful option is <literal>-d</literal> which does not install the
+fetched files. If the system has to download a large number of package it
+would be undesired to start installing them in case something goes wrong. When
+<literal>-d</literal> is used the downloaded archives can be installed by
+simply running the command that caused them to be downloaded again without
+<literal>-d</literal>.
+</para>
+</chapter>
+
+<chapter id="ch3"><title>DSelect</title>
+<para>
+The APT <command>dselect</command> method provides the complete
+APT system with the <command>dselect</command> package selection
+GUI. <command>dselect</command> is used to select the packages to be
+installed or removed and APT actually installs them.
+</para>
+<para>
+To enable the APT method you need to select [A]ccess in
+<command>dselect</command> and then choose the APT method. You will be
+prompted for a set of <emphasis>Sources</emphasis> which are places to fetch
+archives from. These can be remote Internet sites, local Debian mirrors or
+CD-ROMs. Each source can provide a fragment of the total Debian archive, APT
+will automatically combine them to form a complete set of packages. If you
+have a CD-ROM then it is a good idea to specify it first and then specify a
+mirror so that you have access to the latest bug fixes. APT will automatically
+use packages on your CD-ROM before downloading from the Internet.
+</para>
+<screen>
+   Set up a list of distribution source locations
+
+ Please give the base URL of the debian distribution.
+ The access schemes I know about are: http file
+
+ For example:
+      file:/mnt/debian,
+      ftp://ftp.debian.org/debian,
+      http://ftp.de.debian.org/debian,
+
+
+ URL [http://llug.sep.bnl.gov/debian]:
+</screen>
+<para>
+The <emphasis>Sources</emphasis> setup starts by asking for the base of the
+Debian archive, defaulting to a HTTP mirror. Next it asks for the distribution
+to get.
+</para>
+<screen>
+ Please give the distribution tag to get or a path to the
+ package file ending in a /. The distribution
+ tags are typically something like: stable unstable testing non-US
+
+ Distribution [stable]:
+</screen>
+<para>
+The distribution refers to the Debian version in the archive,
+<emphasis>stable</emphasis> refers to the latest released version
+and <emphasis>unstable</emphasis> refers to the developmental
+version. <emphasis>non-US</emphasis> is only available on some mirrors
+and refers to packages that contain encryption technology or other
+things that cannot be exported from the United States. Importing these
+packages into the US is legal however.
+</para>
+<screen>
+ Please give the components to get
+ The components are typically something like: main contrib non-free
+
+ Components [main contrib non-free]:
+</screen>
+<para>
+The components list refers to the list of sub distributions to fetch. The
+distribution is split up based on software licenses, main being DFSG free
+packages while contrib and non-free contain things that have various
+restrictions placed on their use and distribution.
+</para>
+<para>
+Any number of sources can be added, the setup script will continue to prompt
+until you have specified all that you want.
+</para>
+<para>
+Before starting to use <command>dselect</command> it is necessary to update
+the available list by selecting [U]pdate from the menu. This is a superset of
+<literal>apt-get update</literal> that makes the fetched information available
+to <command>dselect</command>. [U]pdate must be performed even if
+<literal>apt-get update</literal> has been run before.
+</para>
+<para>
+You can then go on and make your selections using [S]elect and then perform
+the installation using [I]nstall. When using the APT method the [C]onfig and
+[R]emove commands have no meaning, the [I]nstall command performs both of
+them together.
+</para>
+<para>
+By default APT will automatically remove the package (.deb) files once they
+have been successfully installed. To change this behavior place
+<literal>Dselect::clean "prompt";</literal> in /etc/apt/apt.conf.
+</para>
+</chapter>
+
+<chapter id="ch4"><title>The Interface</title>
+<para>
+Both that APT <command>dselect</command> method and <command>apt-get</command>
+share the same interface. It is a simple system that generally tells you what
+it will do and then goes and does it. <footnote><para> The
+<command>dselect</command> method actually is a set of wrapper scripts to
+<command>apt-get</command>. The method actually provides more functionality
+than is present in <command>apt-get</command> alone. </para> </footnote> After
+printing out a summary of what will happen APT then will print out some
+informative status messages so that you can estimate how far along it is and
+how much is left to do.
+</para>
+
+<section id="s4.1"><title>Startup</title>
+<para>
+Before all operations except update, APT performs a number of actions
+to prepare its internal state. It also does some checks of the system's
+state. At any time these operations can be performed by running
+<literal>apt-get check</literal>.
+</para>
+<screen>
+# apt-get check
+Reading Package Lists... Done
+Building Dependency Tree... Done
+</screen>
+<para>
+The first thing it does is read all the package files into memory. APT uses a
+caching scheme so this operation will be faster the second time it is run. If
+some of the package files are not found then they will be ignored and a
+warning will be printed when apt-get exits.
+</para>
+<para>
+The final operation performs a detailed analysis of the system's
+dependencies. It checks every dependency of every installed or unpacked
+package and considers if it is OK. Should this find a problem then a report
+will be printed out and <command>apt-get</command> will refuse to run.
+</para>
+<screen>
+# apt-get check
+Reading Package Lists... Done
+Building Dependency Tree... Done
+You might want to run apt-get -f install' to correct these.
+Sorry, but the following packages have unmet dependencies:
+  9fonts: Depends: xlib6g but it is not installed
+  uucp: Depends: mailx but it is not installed
+  blast: Depends: xlib6g (&gt;= 3.3-5) but it is not installed
+  adduser: Depends: perl-base but it is not installed
+  aumix: Depends: libgpmg1 but it is not installed
+  debiandoc-sgml: Depends: sgml-base but it is not installed
+  bash-builtins: Depends: bash (&gt;= 2.01) but 2.0-3 is installed
+  cthugha: Depends: svgalibg1 but it is not installed
+           Depends: xlib6g (&gt;= 3.3-5) but it is not installed
+  libreadlineg2: Conflicts:libreadline2 (&lt;&lt; 2.1-2.1)
+</screen>
+<para>
+In this example the system has many problems, including a serious problem with
+libreadlineg2. For each package that has unmet dependencies a line is printed
+out indicating the package with the problem and the dependencies that are
+unmet. A short explanation of why the package has a dependency problem is also
+included.
+</para>
+<para>
+There are two ways a system can get into a broken state like this. The
+first is caused by <command>dpkg</command> missing some subtle relationships
+between packages when performing upgrades. <footnote><para> APT however
+considers all known dependencies and attempts to prevent broken
+packages </para> </footnote>. The second is if a package installation
+fails during an operation. In this situation a package may have been
+unpacked without its dependents being installed.
+</para>
+<para>
+The second situation is much less serious than the first because APT places
+certain constraints on the order that packages are installed. In both cases
+supplying the <literal>-f</literal> option to <command>apt-get</command>
+will cause APT to deduce a possible solution to the problem and then
+continue on. The APT <command>dselect</command> method always supplies
+the <literal>-f</literal> option to allow for easy continuation of failed
+maintainer scripts.
+</para>
+<para>
+However, if the <literal>-f</literal> option is used to correct a seriously
+broken system caused by the first case then it is possible that it will either
+fail immediately or the installation sequence will fail. In either case it is
+necessary to manually use dpkg (possibly with forcing options) to correct the
+situation enough to allow APT to proceed.
+</para>
+</section>
+
+<section id="s4.2"><title>The Status Report</title>
+<para>
+Before proceeding <command>apt-get</command> will present a report on what will
+happen. Generally the report reflects the type of operation being performed
+but there are several common elements. In all cases the lists reflect the
+final state of things, taking into account the <literal>-f</literal> option
+and any other relevant activities to the command being executed.
+</para>
+
+<section id="s4.2.1"><title>The Extra Package list</title>
+<screen>
+The following extra packages will be installed:
+  libdbd-mysql-perl xlib6 zlib1 xzx libreadline2 libdbd-msql-perl
+  mailpgp xdpkg fileutils pinepgp zlib1g xlib6g perl-base
+  bin86 libgdbm1 libgdbmg1 quake-lib gmp2 bcc xbuffy
+  squake pgp-i python-base debmake ldso perl libreadlineg2
+  ssh
+</screen>
+<para>
+The Extra Package list shows all of the packages that will be installed or
+upgraded in excess of the ones mentioned on the command line. It is only
+generated for an <literal>install</literal> command. The listed packages are
+often the result of an Auto Install.
+</para>
+</section>
+
+<section id="s4.2.2"><title>The Packages to Remove</title>
+<screen>
+The following packages will be REMOVED:
+  xlib6-dev xpat2 tk40-dev xkeycaps xbattle xonix
+  xdaliclock tk40 tk41 xforms0.86 ghostview xloadimage xcolorsel
+  xadmin xboard perl-debug tkined xtetris libreadline2-dev perl-suid
+  nas xpilot xfig
+</screen>
+<para>
+The Packages to Remove list shows all of the packages that will be removed
+from the system. It can be shown for any of the operations and should be given
+a careful inspection to ensure nothing important is to be taken off. The
+<literal>-f</literal> option is especially good at generating packages to
+remove so extreme care should be used in that case. The list may contain
+packages that are going to be removed because they are only partially
+installed, possibly due to an aborted installation.
+</para>
+</section>
+
+<section id="s4.2.3"><title>The New Packages list</title>
+<screen>
+The following NEW packages will installed:
+  zlib1g xlib6g perl-base libgdbmg1 quake-lib gmp2 pgp-i python-base
+</screen>
+<para>
+The New Packages list is simply a reminder of what will happen. The packages
+listed are not presently installed in the system but will be when APT is done.
+</para>
+</section>
+
+<section id="s4.2.4"><title>The Kept Back list</title>
+<screen>
+The following packages have been kept back
+  compface man-db tetex-base msql libpaper svgalib1
+  gs snmp arena lynx xpat2 groff xscreensaver
+</screen>
+<para>
+Whenever the whole system is being upgraded there is the possibility that new
+versions of packages cannot be installed because they require new things or
+conflict with already installed things. In this case the package will appear
+in the Kept Back list. The best way to convince packages listed there to
+install is with <literal>apt-get install</literal> or by using
+<command>dselect</command> to resolve their problems.
+</para>
+</section>
+
+<section id="s4.2.5"><title>Held Packages warning</title>
+<screen>
+The following held packages will be changed:
+  cvs
+</screen>
+<para>
+Sometimes you can ask APT to install a package that is on hold, in such a case
+it prints out a warning that the held package is going to be changed. This
+should only happen during dist-upgrade or install.
+</para>
+</section>
+
+<section id="s4.2.6"><title>Final summary</title>
+<para>
+Finally, APT will print out a summary of all the changes that will occur.
+</para>
+<screen>
+206 packages upgraded, 8 newly installed, 23 to remove and 51 not upgraded.
+12 packages not fully installed or removed.
+Need to get 65.7M/66.7M of archives. After unpacking 26.5M will be used.
+</screen>
+<para>
+The first line of the summary simply is a reduced version of all of the lists
+and includes the number of upgrades - that is packages already installed that
+have new versions available. The second line indicates the number of poorly
+configured packages, possibly the result of an aborted installation. The final
+line shows the space requirements that the installation needs. The first pair
+of numbers refer to the size of the archive files. The first number indicates
+the number of bytes that must be fetched from remote locations and the second
+indicates the total size of all the archives required. The next number
+indicates the size difference between the presently installed packages and the
+newly installed packages. It is roughly equivalent to the space required in
+/usr after everything is done. If a large number of packages are being removed
+then the value may indicate the amount of space that will be freed.
+</para>
+<para>
+Some other reports can be generated by using the -u option to show packages to
+upgrade, they are similar to the previous examples.
+</para>
+</section>
+
+</section>
+
+<section id="s4.3"><title>The Status Display</title>
+<para>
+During the download of archives and package files APT prints out a series of
+status messages.
+</para>
+<screen>
+# apt-get update
+Get:1 http://ftp.de.debian.org/debian-non-US/ stable/non-US/ Packages
+Get:2 http://llug.sep.bnl.gov/debian/ testing/contrib Packages
+Hit http://llug.sep.bnl.gov/debian/ testing/main Packages
+Get:4 http://ftp.de.debian.org/debian-non-US/ unstable/binary-i386/ Packages
+Get:5 http://llug.sep.bnl.gov/debian/ testing/non-free Packages
+11% [5 testing/non-free `Waiting for file' 0/32.1k 0%] 2203b/s 1m52s
+</screen>
+<para>
+The lines starting with <emphasis>Get</emphasis> are printed out when APT
+begins to fetch a file while the last line indicates the progress of the
+download. The first percent value on the progress line indicates the total
+percent done of all files. Unfortunately since the size of the Package files
+is unknown <literal>apt-get update</literal> estimates the percent done which
+causes some inaccuracies.
+</para>
+<para>
+The next section of the status line is repeated once for each download
+thread and indicates the operation being performed and some useful
+information about what is happening. Sometimes this section will simply
+read <emphasis>Forking</emphasis> which means the OS is loading the download
+module. The first word after the [ is the fetch number as shown on the
+history lines. The next word is the short form name of the object being
+downloaded. For archives it will contain the name of the package that is
+being fetched.
+</para>
+<para>
+Inside of the single quote is an informative string indicating the progress of
+the negotiation phase of the download. Typically it progresses from
+<emphasis>Connecting</emphasis> to <emphasis>Waiting for file</emphasis> to
+<emphasis>Downloading</emphasis> or <emphasis>Resuming</emphasis>. The final
+value is the number of bytes downloaded from the remote site. Once the
+download begins this is represented as <literal>102/10.2k</literal> indicating
+that 102 bytes have been fetched and 10.2 kilobytes is expected. The total
+size is always shown in 4 figure notation to preserve space. After the size
+display is a percent meter for the file itself. The second last element is the
+instantaneous average speed. This values is updated every 5 seconds and
+reflects the rate of data transfer for that period. Finally is shown the
+estimated transfer time. This is updated regularly and reflects the time to
+complete everything at the shown transfer rate.
+</para>
+<para>
+The status display updates every half second to provide a constant feedback on
+the download progress while the Get lines scroll back whenever a new file is
+started. Since the status display is constantly updated it is unsuitable for
+logging to a file, use the <literal>-q</literal> option to remove the status
+display.
+</para>
+</section>
+
+<section id="s4.4"><title>Dpkg</title>
+<para>
+APT uses <command>dpkg</command> for installing the archives and will
+switch over to the <command>dpkg</command> interface once downloading is
+completed. <command>dpkg</command> will also ask a number of questions as
+it processes the packages and the packages themselves may also ask several
+questions. Before each question there is usually a description of what it
+is asking and the questions are too varied to discuss completely here.
+</para>
+</section>
+
+</chapter>
+
+</book>

doc/guide.sgml

@@ -1,547 +0,0 @@
-<!-- -*- mode: sgml; mode: fold -*- -->
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<book>
-<title>APT User's Guide</title>
-
-<author>Jason Gunthorpe <email>jgg@debian.org</email></author>
-<version>$Id: guide.sgml,v 1.7 2003/04/26 23:26:13 doogie Exp $</version>
-
-<abstract>
-This document provides an overview of how to use the the APT package manager.
-</abstract>
-
-<copyright>
-Copyright &copy; Jason Gunthorpe, 1998.
-<p>
-"APT" and this document are free software; you can redistribute them and/or
-modify them under the terms of the GNU General Public License as published
-by the Free Software Foundation; either version 2 of the License, or (at your
-option) any later version.
-
-<p>
-For more details, on Debian systems, see the file
-/usr/share/common-licenses/GPL for the full license.
-</copyright>
-
-<toc sect>
-
-<!-- General		                                               {{{ -->
-<!-- ===================================================================== -->
-<chapt>General
-
-<p>
-The APT package currently contains two sections, the APT <prgn>dselect</>
-method and the <prgn>apt-get</> command line user interface. Both provide 
-a way to install and remove packages as well as download new packages from 
-the Internet. 
-
-<sect>Anatomy of the Package System
-<p>
-The Debian packaging system has a large amount of information associated with
-each package to help assure that it integrates cleanly and easily into
-the system. The most prominent of its features is the dependency system.
-
-<p>
-The dependency system allows individual programs to make use of shared 
-elements in the system such as libraries. It simplifies placing infrequently 
-used portions of a program in separate packages to reduce the
-number of things the average user is required to install. Also, it allows
-for choices in mail transport agents, X servers and 
-so on.
-
-<p>
-The first step to understanding the dependency system is to grasp the concept
-of a simple dependency. The meaning of a simple dependency is that a package
-requires another package to be installed at the same time to work properly.
-
-<p>
-For instance, mailcrypt is an emacs extension that aids in encrypting email
-with GPG. Without GPGP installed mailcrypt is useless, so mailcrypt has a
-simple dependency on GPG. Also, because it is an emacs extension it has a 
-simple dependency on emacs, without emacs it is completely useless.
-
-<p>
-The other important dependency to understand is a conflicting dependency. It
-means that a package, when installed with another package, will not work and
-may possibly be extremely harmful to the system. As an example consider a
-mail transport agent such as sendmail, exim or qmail. It is not possible
-to have two mail transport agents installed because both need to listen to
-the network to receive mail. Attempting to install two will seriously
-damage the system so all mail transport agents have a conflicting dependency
-with all other mail transport agents.
-
-<p>
-As an added complication there is the possibility for a package to pretend
-to be another package. Consider that exim and sendmail for many intents are
-identical, they both deliver mail and understand a common interface. Hence,
-the package system has a way for them to declare that they are both
-mail-transport-agents. So, exim and sendmail both declare that they provide a
-mail-transport-agent and other packages that need a mail transport agent
-depend on mail-transport-agent. This can add a great deal of confusion when 
-trying to manually fix packages.
-
-<p>
-At any given time a single dependency may be met by packages that are already
-installed or it may not be. APT attempts to help resolve dependency issues
-by providing a number of automatic algorithms that help in selecting packages
-for installation.
-</sect>
-
-</chapt>
-                                                                  <!-- }}} -->
-<!-- apt-get		                                               {{{ -->
-<!-- ===================================================================== -->
-<chapt>apt-get
-
-<p>
-<prgn>apt-get</> provides a simple way to install packages from the command 
-line. Unlike <prgn>dpkg</>, <prgn>apt-get</> does not understand .deb files, 
-it works with the package's proper name and can only install .deb archives from 
-a <em>Source</>.
-
-<p>
-The first <footnote>If you are using an http proxy server you must set the
-http_proxy environment variable first, see sources.list(5)</footnote> thing that 
-should be done before using <prgn>apt-get</> is to fetch the package lists
-from the <em>Sources</> so that it knows what packages are 
-available. This is done with <tt>apt-get update</>. For instance,
-
-<p>
-<example>
-# apt-get update
-Get http://ftp.de.debian.org/debian-non-US/ stable/binary-i386/ Packages
-Get http://llug.sep.bnl.gov/debian/ testing/contrib Packages
-Reading Package Lists... Done
-Building Dependency Tree... Done
-</example>
-
-<p>
-Once updated there are several commands that can be used:
-<taglist>
-<tag>upgrade<item>
-Upgrade will attempt to gently upgrade the whole system. Upgrade will
-never install a new package or remove an existing package, nor will it
-ever upgrade a package that might cause some other package to break.
-This can be used daily to relatively safely upgrade the system. Upgrade
-will list all of the packages that it could not upgrade, this usually
-means that they depend on new packages or conflict with some other package.
-<prgn>dselect</> or <tt>apt-get install</> can be used to force these
-packages to install.
-
-<tag>install<item>
-Install is used to install packages by name. The package is 
-automatically fetched and installed. This can be useful if you already
-know the name of the package to install and do not want to go into a GUI
-to select it. Any number of packages may be passed to install, they will
-all be fetched. Install automatically attempts to resolve dependency problems
-with the listed packages and will print a summary and ask for confirmation
-if anything other than its arguments are changed.
-
-<tag>dist-upgrade<item>
-Dist-upgrade is a complete upgrader designed to simplify upgrading between
-releases of Debian. It uses a sophisticated algorithm to determine the best
-set of packages to install, upgrade and remove to get as much of the system
-to the newest release. In some situations it may be desired to use dist-upgrade
-rather than spend the time manually resolving dependencies in <prgn>dselect</>.
-Once dist-upgrade has completed then <prgn>dselect</> can be used to install
-any packages that may have been left out.
-
-<p>
-It is important to closely look at what dist-upgrade is going to do, its
-decisions may sometimes be quite surprising.
-</taglist>
-
-<p>
-<prgn>apt-get</> has several command line options that are detailed in its
-man page, <manref name="apt-get" section="8">. The most useful option is 
-<tt>-d</> which does not install the fetched files. If the system has to
-download a large number of package it would be undesired to start installing
-them in case something goes wrong. When <tt>-d</> is used the downloaded
-archives can be installed by simply running the command that caused them to
-be downloaded again without <tt>-d</>.
-
-</chapt>
-                                                                  <!-- }}} -->
-<!-- DSelect		                                               {{{ -->
-<!-- ===================================================================== -->
-<chapt>DSelect
-<p>
-The APT <prgn>dselect</> method provides the complete APT system with 
-the <prgn>dselect</> package selection GUI. <prgn>dselect</> is used to
-select the packages to be installed or removed and APT actually installs them.
-
-<p>
-To enable the APT method you need to select [A]ccess in <prgn>dselect</> 
-and then choose the APT method. You will be prompted for a set of 
-<em>Sources</> which are places to fetch archives from. These can be remote
-Internet sites, local Debian mirrors or CD-ROMs. Each source can provide
-a fragment of the total Debian archive, APT will automatically combine them
-to form a complete set of packages. If you have a CD-ROM then it is a good idea
-to specify it first and then specify a mirror so that you have access to
-the latest bug fixes. APT will automatically use packages on your CD-ROM before
-downloading from the Internet.
-
-<p>
-<example>
-   Set up a list of distribution source locations
-	 
- Please give the base URL of the debian distribution.
- The access schemes I know about are: http file
-	   
- For example:
-      file:/mnt/debian,
-      ftp://ftp.debian.org/debian,
-      http://ftp.de.debian.org/debian,
-      
-      
- URL [http://llug.sep.bnl.gov/debian]: 
-</example>
-
-<p>
-The <em>Sources</> setup starts by asking for the base of the Debian
-archive, defaulting to a HTTP mirror. Next it asks for the distribution to
-get.
-
-<p>
-<example>
- Please give the distribution tag to get or a path to the
- package file ending in a /. The distribution
- tags are typically something like: stable unstable testing non-US
-   
- Distribution [stable]: 
-</example>
-
-<p>
-The distribution refers to the Debian version in the archive, <em>stable</>
-refers to the latest released version and <em>unstable</> refers to the
-developmental version. <em>non-US</> is only available on some mirrors and
-refers to packages that contain encryption technology or other things that
-cannot be exported from the United States. Importing these packages into the
-US is legal however. 
-
-<p>
-<example>
- Please give the components to get
- The components are typically something like: main contrib non-free
-  
- Components [main contrib non-free]:
-</example>
-
-<p>
-The components list refers to the list of sub distributions to fetch. The
-distribution is split up based on software licenses, main being DFSG free
-packages while contrib and non-free contain things that have various 
-restrictions placed on their use and distribution.
-
-<p>
-Any number of sources can be added, the setup script will continue to
-prompt until you have specified all that you want.
-
-<p>
-Before starting to use <prgn>dselect</> it is necessary to update the 
-available list by selecting [U]pdate from the menu. This is a superset of 
-<tt>apt-get update</> that makes the fetched information available to
-<prgn>dselect</>. [U]pdate must be performed even if <tt>apt-get update</>
-has been run before.
-
-<p>
-You can then go on and make your selections using [S]elect and then 
-perform the installation using [I]nstall. When using the APT method
-the [C]onfig and [R]emove commands have no meaning, the [I]nstall command
-performs both of them together. 
-
-<p>
-By default APT will automatically remove the package (.deb) files once they have been
-successfully installed. To change this behavior place <tt>Dselect::clean 
-"prompt";</> in /etc/apt/apt.conf.
-
-</chapt>
-                                                                  <!-- }}} -->
-<!-- The Interfaces						       {{{ -->
-<!-- ===================================================================== -->
-<chapt>The Interface
-
-<p>
-Both that APT <prgn>dselect</> method and <prgn>apt-get</> share the same
-interface. It is a simple system that generally tells you what it will do
-and then goes and does it. 
-<footnote>
-The <prgn>dselect</> method actually is a set of wrapper scripts
-to <prgn>apt-get</>. The method actually provides more functionality than
-is present in <prgn>apt-get</> alone.
-</footnote>
-After printing out a summary of what will happen APT then will print out some
-informative status messages so that you can estimate how far along it is and
-how much is left to do.
-
-<!-- ===================================================================== -->
-<sect>Startup
-
-<p>
-Before all operations except update, APT performs a number of actions to
-prepare its internal state. It also does some checks of the system's state.
-At any time these operations can be performed by running <tt>apt-get check</>.
-<p>
-<example>
-# apt-get check
-Reading Package Lists... Done
-Building Dependency Tree... Done
-</example>
-
-<p>
-The first thing it does is read all the package files into memory. APT
-uses a caching scheme so this operation will be faster the second time it
-is run. If some of the package files are not found then they will be ignored
-and a warning will be printed when apt-get exits. 
-
-<p>
-The final operation performs a detailed analysis of the system's dependencies.
-It checks every dependency of every installed or unpacked package and considers
-if it is OK. Should this find a problem then a report will be printed out and
-<prgn>apt-get</> will refuse to run.
-
-<p>
-<example>
-# apt-get check
-Reading Package Lists... Done
-Building Dependency Tree... Done
-You might want to run apt-get -f install' to correct these.
-Sorry, but the following packages have unmet dependencies:
-  9fonts: Depends: xlib6g but it is not installed
-  uucp: Depends: mailx but it is not installed
-  blast: Depends: xlib6g (>= 3.3-5) but it is not installed
-  adduser: Depends: perl-base but it is not installed
-  aumix: Depends: libgpmg1 but it is not installed
-  debiandoc-sgml: Depends: sgml-base but it is not installed
-  bash-builtins: Depends: bash (>= 2.01) but 2.0-3 is installed
-  cthugha: Depends: svgalibg1 but it is not installed
-           Depends: xlib6g (>= 3.3-5) but it is not installed
-  libreadlineg2: Conflicts:libreadline2 (<< 2.1-2.1)
-</example>
-
-<p>
-In this example the system has many problems, including a serious problem
-with libreadlineg2. For each package that has unmet dependencies a line
-is printed out indicating the package with the problem and the dependencies
-that are unmet. A short explanation of why the package has a dependency
-problem is also included.
-
-<p>
-There are two ways a system can get into a broken state like this. The
-first is caused by <prgn>dpkg</> missing some subtle relationships between 
-packages when performing upgrades. <footnote>APT however considers all known 
-dependencies and attempts to prevent broken packages</footnote>. The second is 
-if a package installation fails during an operation. In this situation a 
-package may have been unpacked without its dependents being installed.
-
-<p>
-The second situation is much less serious than the first because APT places
-certain constraints on the order that packages are installed. In both cases
-supplying the <tt>-f</> option to <prgn>apt-get</> will cause APT to deduce a
-possible solution to the problem and then continue on. The APT <prgn>dselect</> 
-method always supplies the <tt>-f</> option to allow for easy continuation
-of failed maintainer scripts.
-
-<p>
-However, if the <tt>-f</> option is used to correct a seriously broken system
-caused by the first case then it is possible that it will either fail 
-immediately or the installation sequence will fail. In either case it is
-necessary to manually use dpkg (possibly with forcing options) to correct
-the situation enough to allow APT to proceed.
-</sect>
-
-<!-- ===================================================================== -->
-<sect>The Status Report
-
-<p>
-Before proceeding <prgn>apt-get</> will present a report on what will happen.
-Generally the report reflects the type of operation being performed but there
-are several common elements. In all cases the lists reflect the final state
-of things, taking into account the <tt>-f</> option and any other relevant
-activities to the command being executed.
-
-<sect1>The Extra Package list
-<p>
-<example>
-The following extra packages will be installed:
-  libdbd-mysql-perl xlib6 zlib1 xzx libreadline2 libdbd-msql-perl
-  mailpgp xdpkg fileutils pinepgp zlib1g xlib6g perl-base
-  bin86 libgdbm1 libgdbmg1 quake-lib gmp2 bcc xbuffy
-  squake pgp-i python-base debmake ldso perl libreadlineg2
-  ssh
-</example>
-
-<p>
-The Extra Package list shows all of the packages that will be installed 
-or upgraded in excess of the ones mentioned on the command line. It is
-only generated for an <tt>install</> command. The listed packages are
-often the result of an Auto Install.
-</sect1>
-
-<sect1>The Packages to Remove
-<p>
-<example>
-The following packages will be REMOVED:
-  xlib6-dev xpat2 tk40-dev xkeycaps xbattle xonix
-  xdaliclock tk40 tk41 xforms0.86 ghostview xloadimage xcolorsel
-  xadmin xboard perl-debug tkined xtetris libreadline2-dev perl-suid
-  nas xpilot xfig 
-</example>
-
-<p>
-The Packages to Remove list shows all of the packages that will be
-removed from the system. It can be shown for any of the operations and 
-should be given a careful inspection to ensure nothing important is to 
-be taken off. The <tt>-f</> option is especially good at generating packages
-to remove so extreme care should be used in that case. The list may contain
-packages that are going to be removed because they are only 
-partially installed, possibly due to an aborted installation.
-</sect1>
-
-<sect1>The New Packages list
-<p>
-<example>
-The following NEW packages will installed:
-  zlib1g xlib6g perl-base libgdbmg1 quake-lib gmp2 pgp-i python-base
-</example>
-
-<p>
-The New Packages list is simply a reminder of what will happen. The packages
-listed are not presently installed in the system but will be when APT is done.
-</sect1>
-
-<sect1>The Kept Back list
-<p>
-<example>
-The following packages have been kept back
-  compface man-db tetex-base msql libpaper svgalib1
-  gs snmp arena lynx xpat2 groff xscreensaver
-</example>
-
-<p>
-Whenever the whole system is being upgraded there is the possibility that
-new versions of packages cannot be installed because they require new things
-or conflict with already installed things. In this case the package will 
-appear in the Kept Back list. The best way to convince packages listed
-there to install is with <tt>apt-get install</> or by using <prgn>dselect</>
-to resolve their problems.
-</sect1>
-
-<sect1>Held Packages warning
-<p>
-<example>
-The following held packages will be changed:
-  cvs 
-</example>
-
-<p>
-Sometimes you can ask APT to install a package that is on hold, in such a 
-case it prints out a warning that the held package is going to be
-changed. This should only happen during dist-upgrade or install.
-</sect1>
-
-<sect1>Final summary
-<p>
-Finally, APT will print out a summary of all the changes that will occur.
-
-<p>
-<example>
-206 packages upgraded, 8 newly installed, 23 to remove and 51 not upgraded.
-12 packages not fully installed or removed.
-Need to get 65.7M/66.7M of archives. After unpacking 26.5M will be used. 
-</example>
-
-<p>
-The first line of the summary simply is a reduced version of all of the
-lists and includes the number of upgrades - that is packages already 
-installed that have new versions available. The second line indicates the
-number of poorly configured packages, possibly the result of an aborted
-installation. The final line shows the space requirements that the
-installation needs. The first pair of numbers refer to the size of
-the archive files. The first number indicates the number of bytes that
-must be fetched from remote locations and the second indicates the
-total size of all the archives required. The next number indicates the
-size difference between the presently installed packages and the newly
-installed packages. It is roughly equivalent to the space required in 
-/usr after everything is done. If a large number of packages are being
-removed then the value may indicate the amount of space that will be
-freed.
-
-<p>
-Some other reports can be generated by using the -u option to show packages
-to upgrade, they are similar to the previous examples.
-</sect>
-
-<!-- ===================================================================== -->
-<sect>The Status Display
-<p>
-During the download of archives and package files APT prints out a series of
-status messages.
-
-<p>
-<example>
-# apt-get update
-Get:1 http://ftp.de.debian.org/debian-non-US/ stable/non-US/ Packages
-Get:2 http://llug.sep.bnl.gov/debian/ testing/contrib Packages
-Hit http://llug.sep.bnl.gov/debian/ testing/main Packages
-Get:4 http://ftp.de.debian.org/debian-non-US/ unstable/binary-i386/ Packages
-Get:5 http://llug.sep.bnl.gov/debian/ testing/non-free Packages
-11% [5 testing/non-free `Waiting for file' 0/32.1k 0%] 2203b/s 1m52s
-</example>
-
-<p>
-The lines starting with <em>Get</> are printed out when APT begins to fetch 
-a file while the last line indicates the progress of the download. The first 
-percent  value on the progress line indicates the total percent done of all 
-files. Unfortunately since the size of the Package files is unknown 
-<tt>apt-get update</> estimates the percent done which causes some 
-inaccuracies.
-
-<p>
-The next section of the status line is repeated once for each download thread
-and indicates the operation being performed and some useful information
-about what is happening. Sometimes this section will simply read <em>Forking</>
-which means the OS is loading the download module. The first word after the [
-is the fetch number as shown on the history lines. The next word
-is the short form name of the object being downloaded. For archives it will
-contain the name of the package that is being fetched.
-
-<p>
-Inside of the single quote is an informative string indicating the progress
-of the negotiation phase of the download. Typically it progresses from 
-<em>Connecting</> to <em>Waiting for file</> to <em>Downloading</> or
-<em>Resuming</>. The final value is the number of bytes downloaded from the
-remote site. Once the download begins this is represented as <tt>102/10.2k</>
-indicating that 102 bytes have been fetched and 10.2 kilobytes is expected.
-The total size is always shown in 4 figure notation to preserve space. After
-the size display is a percent meter for the file itself.
-The second last element is the instantaneous average speed. This values is 
-updated every 5 seconds and reflects the rate of data transfer for that 
-period. Finally is shown the estimated transfer time. This is updated
-regularly and reflects the time to complete everything at the shown 
-transfer rate.
-
-<p> 
-The status display updates every half second to provide a constant feedback 
-on the download progress while the Get lines scroll back whenever a new
-file is started. Since the status display is constantly updated it is
-unsuitable for logging to a file, use the <tt>-q</> option to remove the
-status display.
-</sect>
-
-<!-- ===================================================================== -->
-<sect>Dpkg
-
-<p>
-APT uses <prgn>dpkg</> for installing the archives and will switch
-over to the <prgn>dpkg</> interface once downloading is completed.
-<prgn>dpkg</> will also ask a number of questions as it processes the packages
-and the packages themselves may also ask several questions. Before each 
-question there is usually a description of what it is asking and the
-questions are too varied to discuss completely here.
-</sect>
-
-</chapt>
-                                                                  <!-- }}} -->
-
-</book>

doc/makefile

@@ -5,12 +5,12 @@ SUBDIR=doc
 # Bring in the default rules
 include ../buildlib/defaults.mak
 
-# Debian Doc SGML Documents
-SOURCE = $(wildcard *.sgml)
-DEBIANDOC_HTML_OPTIONS=-l en.UTF-8
-include $(DEBIANDOC_H)
+# DocBook XML Documents
+SOURCE = $(wildcard *.dbk)
+LC = en
+include $(DOCBOOK_H)
 
-doc: manpages debiandoc
+doc: manpages docbook
 
 examples/sources.list: ../vendor/current/sources.list
 	ln -sf $(shell readlink -f $^) $@
@@ -24,12 +24,12 @@ TO = $(DOC)
 TARGET = binary
 include $(COPY_H)
 
-.PHONY: clean clean/subdirs veryclean veryclean/subdirs manpages/subdirs debiandoc/subdirs all binary doc stats
+.PHONY: clean clean/subdirs veryclean veryclean/subdirs manpages/subdirs docbook/subdirs all binary doc stats
 
 clean: clean/subdirs clean/examples
 veryclean: veryclean/subdirs clean/examples
 manpages: apt-vendor.ent manpages/subdirs
-debiandoc: debiandoc/subdirs
+docbook: apt-vendor.ent docbook/subdirs
 
 DOCUMENTATIONPO = $(patsubst %.po,%,$(notdir $(wildcard po/*.po)))
 DOCDIRLIST = $(addsuffix /makefile,$(DOCUMENTATIONPO))
@@ -39,7 +39,7 @@ $(DOCDIRLIST) :: %/makefile : lang.makefile
 	test -d $(dir $@) || mkdir $(dir $@)
 	sed "s#@@LANG@@#$(subst /,,$(dir $@))#" $< > $@
 
-debiandoc/subdirs manpages/subdirs clean/subdirs veryclean/subdirs:
+docbook/subdirs manpages/subdirs clean/subdirs veryclean/subdirs:
 	for dir in en $(dir $(DOCDIRLIST)); do \
 		$(MAKE) -C $$dir $(patsubst %/subdirs,%,$@); \
 	done
@@ -53,11 +53,11 @@ stats:
 
 ifdef PO4A
 MANPAGEPOLIST = $(addprefix manpages-translation-,$(DOCUMENTATIONPO))
-DEBIANDOCPOLIST = $(addprefix debiandoc-translation-,$(DOCUMENTATIONPO))
+DOCBOOKPOLIST = $(addprefix docbook-translation-,$(DOCUMENTATIONPO))
 
-.PHONY: update-po po4a $(MANPAGEPOLIST) $(DEBIANDOCPOLIST) $(DOCDIRLIST)
+.PHONY: update-po po4a $(MANPAGEPOLIST) $(DOCBOOKPOLIST) $(DOCDIRLIST)
 
-po4a: manpages/subdirs debiandoc/subdirs
+po4a: manpages/subdirs docbook/subdirs
 
 update-po:
 	po4a --previous --no-backups --force --no-translations \
@@ -78,10 +78,10 @@ $(MANPAGEPOLIST) :: manpages-translation-% : %/makefile po4a.conf
 		--package-name='$(PACKAGE)-doc' --package-version='$(PACKAGE_VERSION)' \
 		--msgid-bugs-address='$(PACKAGE_MAIL)' po4a.conf
 
-debiandoc/subdirs: $(DEBIANDOCPOLIST)
-$(DEBIANDOCPOLIST) :: debiandoc-translation-% : %/makefile po4a.conf
+docbook/subdirs: $(DOCBOOKPOLIST)
+$(DOCBOOKPOLIST) :: docbook-translation-% : %/makefile po4a.conf
 	po4a --previous --no-backups --translate-only $(dir $<)apt.ent \
-		$(patsubst %,--translate-only $(dir $<)%,$(patsubst %.sgml,%.$(subst /,,$(dir $<)).sgml,$(wildcard *.sgml))) \
+		$(patsubst %,--translate-only $(dir $<)%,$(patsubst %.dbk,%.$(subst /,,$(dir $<)).dbk,$(wildcard *.dbk))) \
 		--package-name='$(PACKAGE)-doc' --package-version='$(PACKAGE_VERSION)' \
 		--msgid-bugs-address='$(PACKAGE_MAIL)' po4a.conf
 endif
@@ -101,5 +101,5 @@ $(BUILD)/doc/doxygen-stamp: $(DOXYGEN_SOURCES) $(BUILD)/doc/Doxyfile
 	$(DOXYGEN) $(BUILD)/doc/Doxyfile
 	touch $(BUILD)/doc/doxygen-stamp
 
-debiandoc: $(BUILD)/doc/doxygen-stamp
+docbook: $(BUILD)/doc/doxygen-stamp
 endif

doc/manpage-style.xsl

@@ -5,7 +5,6 @@
 <xsl:import href="/usr/share/xml/docbook/stylesheet/nwalsh/manpages/docbook.xsl" />
 
 <xsl:param name="man.output.encoding" select="'UTF-8'" />
-<!-- LANGUAGE -->
 
 <xsl:template match="email">&lt;<xsl:apply-templates/>&gt;</xsl:template>
 

doc/method.dbk

@@ -0,0 +1,713 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
+<title>APT Method Interface</title>
+
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Jason Gunthorpe</personname><email>jgg@debian.org</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
+
+<abstract>
+<para>
+This document describes the interface that APT uses to the archive access
+methods.
+</para>
+</abstract>
+
+<copyright><year>1998</year><holder>Jason Gunthorpe</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
+"APT" and this document are free software; you can redistribute them and/or
+modify them under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at your
+option) any later version.
+</para>
+<para>
+For more details, on Debian systems, see the file
+/usr/share/common-licenses/GPL for the full license.
+</para>
+</legalnotice>
+
+</bookinfo>
+
+<chapter id="ch1"><title>Introduction</title>
+
+<section id="s1.1"><title>General</title>
+<para>
+The APT method interface allows APT to acquire archive files (.deb), index
+files (Packages, Release, Mirrors) and source files (.tar.gz, .diff). It is a
+general, extensible system designed to satisfy all of these requirements:
+</para>
+<orderedlist numeration="arabic">
+<listitem>
+<para>
+Remote methods that download files from a distant site
+</para>
+</listitem>
+<listitem>
+<para>
+Resume of aborted downloads
+</para>
+</listitem>
+<listitem>
+<para>
+Progress reporting
+</para>
+</listitem>
+<listitem>
+<para>
+If-Modified-Since (IMS) checking for index files
+</para>
+</listitem>
+<listitem>
+<para>
+In-Line MD5 generation
+</para>
+</listitem>
+<listitem>
+<para>
+No-copy in-filesystem methods
+</para>
+</listitem>
+<listitem>
+<para>
+Multi-media methods (like CD's)
+</para>
+</listitem>
+<listitem>
+<para>
+Dynamic source selection for failure recovery
+</para>
+</listitem>
+<listitem>
+<para>
+User interaction for user/password requests and media swaps
+</para>
+</listitem>
+<listitem>
+<para>
+Global configuration
+</para>
+</listitem>
+</orderedlist>
+<para>
+Initial releases of APT (0.1.x) used a completely different method interface
+that only supported the first 6 items. This new interface deals with the
+remainder.
+</para>
+</section>
+
+<section id="s1.2"><title>Terms</title>
+<para>
+Several terms are used through out the document, they have specific meanings
+which may not be immediately evident. To clarify they are summarized here.
+</para>
+<variablelist>
+<varlistentry>
+<term>source</term>
+<listitem>
+<para>
+Refers to an item in source list. More specifically it is the broken down
+item, that is each source maps to exactly one index file. Archive sources map
+to Package files and Source Code sources map to Source files.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>archive file</term>
+<listitem>
+<para>
+Refers to a binary package archive (.deb, .rpm, etc).
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>source file</term>
+<listitem>
+<para>
+Refers to one of the files making up the source code of a package. In debian
+it is one of .diff.gz, .dsc. or .tar.gz.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>URI</term>
+<listitem>
+<para>
+Universal Resource Identifier (URI) is a super-set of the familiar URL
+syntax used by web browsers. It consists of an access specification
+followed by a specific location in that access space. The form is
+&lt;access&gt;:&lt;location&gt;. Network addresses are given with the form
+&lt;access&gt;://[&lt;user&gt;[:&lt;pas&gt;]@]hostname[:port]/&lt;location&gt;.
+Some examples:
+</para>
+<screen>
+file:/var/mirrors/debian/
+ftp://ftp.debian.org/debian
+ftp://jgg:MooCow@localhost:21/debian
+nfs://bigred/var/mirrors/debian
+rsync://debian.midco.net/debian
+cdrom:Debian 2.0r1 Disk 1/
+</screen>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>method</term>
+<listitem>
+<para>
+There is a one to one mapping of URI access specifiers to methods. A method is
+a program that knows how to handle a URI access type and operates according to
+the specifications in this file.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>method instance</term>
+<listitem>
+<para>
+A specific running method. There can be more than one instance of each method
+as APT is capable of concurrent method handling.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>message</term>
+<listitem>
+<para>
+A series of lines terminated by a blank line sent down one of the communication
+lines. The first line should have the form xxx TAG where xxx are digits
+forming the status code and TAG is an informational string
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>acquire</term>
+<listitem>
+<para>
+The act of bring a URI into the local pathname space. This may simply be
+verifying the existence of the URI or actually downloading it from a remote
+site.
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</section>
+
+</chapter>
+
+<chapter id="ch2"><title>Specification</title>
+
+<section id="s2.1"><title>Overview</title>
+<para>
+All methods operate as a sub process of a main controlling parent. 3 FD's are
+opened for use by the method allowing two way communication and emergency error
+reporting. The FD's correspond to the well known unix FD's, stdin, stdout and
+stderr.
+</para>
+<para>
+Through operation of the method communication is done via http style plain
+text. Specifically RFC-822 (like the Package file) fields are used to describe
+items and a numeric-like header is used to indicate what is happening. Each of
+these distinct communication messages should be sent quickly and without pause.
+</para>
+<para>
+In some instances APT may pre-invoke a method to allow things like file URI's
+to determine how many files are available locally.
+</para>
+</section>
+
+<section id="s2.2"><title>Message Overview</title>
+<para>
+The first line of each message is called the message header. The first 3
+digits (called the Status Code) have the usual meaning found in the http
+protocol. 1xx is informational, 2xx is successful and 4xx is failure. The 6xx
+series is used to specify things sent to the method. After the status code is
+an informational string provided for visual debugging.
+</para>
+<itemizedlist>
+<listitem>
+<para>
+100 Capabilities - Method capabilities
+</para>
+</listitem>
+<listitem>
+<para>
+101 Log - General Logging
+</para>
+</listitem>
+<listitem>
+<para>
+102 Status - Inter-URI status reporting (login progress)
+</para>
+</listitem>
+<listitem>
+<para>
+200 URI Start - URI is starting acquire
+</para>
+</listitem>
+<listitem>
+<para>
+201 URI Done - URI is finished acquire
+</para>
+</listitem>
+<listitem>
+<para>
+400 URI Failure - URI has failed to acquire
+</para>
+</listitem>
+<listitem>
+<para>
+401 General Failure - Method did not like something sent to it
+</para>
+</listitem>
+<listitem>
+<para>
+402 Authorization Required - Method requires authorization to access the URI.
+Authorization is User/Pass
+</para>
+</listitem>
+<listitem>
+<para>
+403 Media Failure - Method requires a media change
+</para>
+</listitem>
+<listitem>
+<para>
+600 URI Acquire - Request a URI be acquired
+</para>
+</listitem>
+<listitem>
+<para>
+601 Configuration - Sends the configuration space
+</para>
+</listitem>
+<listitem>
+<para>
+602 Authorization Credentials - Response to the 402 message
+</para>
+</listitem>
+<listitem>
+<para>
+603 Media Changed - Response to the 403 message
+</para>
+</listitem>
+</itemizedlist>
+<para>
+Only the 6xx series of status codes is sent TO the method. Furthermore the
+method may not emit status codes in the 6xx range. The Codes 402 and 403
+require that the method continue reading all other 6xx codes until the proper
+602/603 code is received. This means the method must be capable of handling an
+unlimited number of 600 messages.
+</para>
+<para>
+The flow of messages starts with the method sending out a <emphasis>100
+Capabilities</emphasis> and APT sending out a <emphasis>601
+Configuration</emphasis>. After that APT begins sending <emphasis>600 URI
+Acquire</emphasis> and the method sends out <emphasis>200 URI Start</emphasis>,
+<emphasis>201 URI Done</emphasis> or <emphasis>400 URI Failure</emphasis>. No
+synchronization is performed, it is expected that APT will send <emphasis>600
+URI Acquire</emphasis> messages at -any- time and that the method should queue
+the messages. This allows methods like http to pipeline requests to the remote
+server. It should be noted however that APT will buffer messages so it is not
+necessary for the method to be constantly ready to receive them.
+</para>
+</section>
+
+<section id="s2.3"><title>Header Fields</title>
+<para>
+The following is a short index of the header fields that are supported
+</para>
+<variablelist>
+<varlistentry>
+<term>URI</term>
+<listitem>
+<para>
+URI being described by the message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Filename</term>
+<listitem>
+<para>
+Location in the filesystem
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Last-Modified</term>
+<listitem>
+<para>
+A time stamp in RFC1123 notation for use by IMS checks
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>IMS-Hit</term>
+<listitem>
+<para>
+The already existing item is valid
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Size</term>
+<listitem>
+<para>
+Size of the file in bytes
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Resume-Point</term>
+<listitem>
+<para>
+Location that transfer was started
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>MD5-Hash</term>
+<listitem>
+<para>
+Computed MD5 hash for the file
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Message</term>
+<listitem>
+<para>
+String indicating some displayable message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Media</term>
+<listitem>
+<para>
+String indicating the media name required
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Site</term>
+<listitem>
+<para>
+String indicating the site authorization is required for
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>User</term>
+<listitem>
+<para>
+Username for authorization
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Password</term>
+<listitem>
+<para>
+Password for authorization
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Fail</term>
+<listitem>
+<para>
+Operation failed
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Drive</term>
+<listitem>
+<para>
+Drive the media should be placed in
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Config-Item</term>
+<listitem>
+<para>
+A string of the form
+<replaceable>item</replaceable>=<replaceable>value</replaceable> derived from
+the APT configuration space. These may include method specific values and
+general values not related to the method. It is up to the method to filter out
+the ones it wants.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Single-Instance</term>
+<listitem>
+<para>
+Requires that only one instance of the method be run This is a yes/no value.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Pipeline</term>
+<listitem>
+<para>
+The method is capable of pipelining.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Local</term>
+<listitem>
+<para>
+The method only returns Filename: fields.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Send-Config</term>
+<listitem>
+<para>
+Send configuration to the method.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Needs-Cleanup</term>
+<listitem>
+<para>
+The process is kept around while the files it returned are being used. This is
+primarily intended for CD-ROM and File URIs that need to unmount filesystems.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>Version</term>
+<listitem>
+<para>
+Version string for the method
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+<para>
+This is a list of which headers each status code can use
+</para>
+<variablelist>
+<varlistentry>
+<term>100 Capabilities</term>
+<listitem>
+<para>
+Displays the capabilities of the method. Methods should set the pipeline bit
+if their underlying protocol supports pipelining. The only known method that
+does support pipelining is http. Fields: Version, Single-Instance, Pre-Scan,
+Pipeline, Send-Config, Needs-Cleanup
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>101 Log</term>
+<listitem>
+<para>
+A log message may be printed to the screen if debugging is enabled. This is
+only for debugging the method. Fields: Message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>102 Status</term>
+<listitem>
+<para>
+Message gives a progress indication for the method. It can be used to show
+pre-transfer status for Internet type methods. Fields: Message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>200 URI Start</term>
+<listitem>
+<para>
+Indicates the URI is starting to be transferred. The URI is specified along
+with stats about the file itself. Fields: URI, Size, Last-Modified,
+Resume-Point
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>201 URI Done</term>
+<listitem>
+<para>
+Indicates that a URI has completed being transferred. It is possible to
+specify a <emphasis>201 URI Done</emphasis> without a <emphasis>URI
+Start</emphasis> which would mean no data was transferred but the file is now
+available. A Filename field is specified when the URI is directly available in
+the local pathname space. APT will either directly use that file or copy it
+into another location. It is possible to return Alt-* fields to indicate that
+another possibility for the URI has been found in the local pathname space.
+This is done if a decompressed version of a .gz file is found. Fields: URI,
+Size, Last-Modified, Filename, MD5-Hash
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>400 URI Failure</term>
+<listitem>
+<para>
+Indicates a fatal URI failure. The URI is not retrievable from this source. As
+with <emphasis>201 URI Done</emphasis> <emphasis>200 URI Start</emphasis> is
+not required to precede this message Fields: URI, Message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>401 General Failure</term>
+<listitem>
+<para>
+Indicates that some unspecific failure has occurred and the method is unable
+to  continue. The method should terminate after sending this message. It
+is intended to check for invalid configuration options or other severe
+conditions. Fields: Message
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>402 Authorization Required</term>
+<listitem>
+<para>
+The method requires a Username and Password pair to continue. After sending
+this message the method will expect APT to send a <emphasis>602 Authorization
+Credentials</emphasis> message with the required information. It is possible
+for a method to send this multiple times. Fields: Site
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>403 Media Failure</term>
+<listitem>
+<para>
+A method that deals with multiple media requires that a new media be
+inserted. The Media field contains the name of the media to be
+inserted. Fields: Media, Drive
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>600 URI Acquire</term>
+<listitem>
+<para>
+APT is requesting that a new URI be added to the acquire list. Last-Modified
+has the time stamp of the currently cache file if applicable. Filename is the
+name of the file that the acquired URI should be written to. Fields: URI,
+Filename Last-Modified
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>601 Configuration</term>
+<listitem>
+<para>
+APT is sending the configuration space to the method. A series of Config-Item
+fields will be part of this message, each containing an entry from the
+configuration space. Fields: Config-Item.
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>602 Authorization Credentials</term>
+<listitem>
+<para>
+This is sent in response to a <emphasis>402 Authorization Required</emphasis>
+message. It contains the entered username and password. Fields: Site, User,
+Password
+</para>
+</listitem>
+</varlistentry>
+<varlistentry>
+<term>603 Media Changed</term>
+<listitem>
+<para>
+This is sent in response to a <emphasis>403 Media Failure</emphasis>
+message. It indicates that the user has changed media and it is safe
+to proceed. Fields: Media, Fail
+</para>
+</listitem>
+</varlistentry>
+</variablelist>
+</section>
+
+<section id="s2.4"><title>Notes</title>
+<para>
+The methods supplied by the stock apt are:
+</para>
+<orderedlist numeration="arabic">
+<listitem>
+<para>
+cdrom - For Multi-Disc CD-ROMs
+</para>
+</listitem>
+<listitem>
+<para>
+copy - (internal) For copying files around the filesystem
+</para>
+</listitem>
+<listitem>
+<para>
+file - For local files
+</para>
+</listitem>
+<listitem>
+<para>
+gzip - (internal) For decompression
+</para>
+</listitem>
+<listitem>
+<para>
+http - For HTTP servers
+</para>
+</listitem>
+</orderedlist>
+<para>
+The two internal methods, copy and gzip, are used by the acquire code to
+parallize and simplify the automatic decompression of package files as well as
+copying package files around the file system. Both methods can be seen to act
+the same except that one decompresses on the fly. APT uses them by generating
+a copy URI that is formed identically to a file URI. The destination file is
+send as normal. The method then takes the file specified by the URI and writes
+it to the destination file. A typical set of operations may be:
+</para>
+<screen>
+http://foo.com/Packages.gz -&gt; /bar/Packages.gz
+gzip:/bar/Packages.gz -&gt; /bar/Packages.decomp
+rename Packages.decomp to /final/Packages
+</screen>
+<para>
+The http method implements a fully featured HTTP/1.1 client that supports
+deep pipelining and reget. It works best when coupled with an apache 1.3
+server. The file method simply generates failures or success responses
+with the filename field set to the proper location. The cdrom method acts
+the same except that it checks that the mount point has a valid cdrom in
+it. It does this by (effectively) computing a md5 hash of 'ls -l' on the
+mountpoint.
+</para>
+</section>
+
+</chapter>
+
+</book>

doc/method.sgml

@@ -1,354 +0,0 @@
-<!-- -*- mode: sgml; mode: fold -*- -->
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<book>
-<title>APT Method Interface </title>
-
-<author>Jason Gunthorpe <email>jgg@debian.org</email></author>
-<version>$Id: method.sgml,v 1.10 2003/02/12 15:05:46 doogie Exp $</version>
-
-<abstract>
-This document describes the interface that APT uses to the archive
-access methods.
-</abstract>
-
-<copyright>
-Copyright &copy; Jason Gunthorpe, 1998.
-<p>
-"APT" and this document are free software; you can redistribute them and/or
-modify them under the terms of the GNU General Public License as published
-by the Free Software Foundation; either version 2 of the License, or (at your
-option) any later version.
-
-<p>
-For more details, on Debian systems, see the file
-/usr/share/common-licenses/GPL for the full license.
-</copyright>
-
-<toc sect>
-
-<chapt>Introduction
-<!-- General		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>General
-
-<p>
-The APT method interface allows APT to acquire archive files (.deb), index
-files (Packages, Release, Mirrors) and source files (.tar.gz, .diff). It
-is a general, extensible system designed to satisfy all of these
-requirements:
-
-<enumlist>
-<item>Remote methods that download files from a distant site
-<item>Resume of aborted downloads
-<item>Progress reporting
-<item>If-Modified-Since (IMS) checking for index files
-<item>In-Line MD5 generation
-<item>No-copy in-filesystem methods
-<item>Multi-media methods (like CD's)
-<item>Dynamic source selection for failure recovery
-<item>User interaction for user/password requests and media swaps
-<item>Global configuration
-</enumlist>
-
-Initial releases of APT (0.1.x) used a completely different method
-interface that only supported the first 6 items. This new interface
-deals with the remainder.
-</sect>
-                                                                  <!-- }}} -->
-<!-- Terms		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Terms
-
-<p>
-Several terms are used through out the document, they have specific
-meanings which may not be immediately evident. To clarify they are summarized
-here.
-
-<taglist>
-<tag>source<item>
-Refers to an item in source list. More specifically it is the broken down
-item, that is each source maps to exactly one index file. Archive sources
-map to Package files and Source Code sources map to Source files.
-
-<tag>archive file<item>
-Refers to a binary package archive (.deb, .rpm, etc). 
-
-<tag>source file<item>
-Refers to one of the files making up the source code of a package. In
-debian it is one of .diff.gz, .dsc. or .tar.gz.
-
-<tag>URI<item>
-Universal Resource Identifier (URI) is a super-set of the familiar URL
-syntax used by web browsers. It consists of an access specification
-followed by a specific location in that access space. The form is
-&lt;access&gt;:&lt;location&gt;. Network addresses are given with the form 
-&lt;access&gt;://[&lt;user&gt;[:&lt;pas&gt;]@]hostname[:port]/&lt;location&gt;. 
-Some examples:
-<example>
-file:/var/mirrors/debian/
-ftp://ftp.debian.org/debian
-ftp://jgg:MooCow@localhost:21/debian
-nfs://bigred/var/mirrors/debian
-rsync://debian.midco.net/debian
-cdrom:Debian 2.0r1 Disk 1/
-</example>
-
-<tag>method<item>
-There is a one to one mapping of URI access specifiers to methods. A method
-is a program that knows how to handle a URI access type and operates according
-to the specifications in this file.
-
-<tag>method instance<item>
-A specific running method. There can be more than one instance of each method
-as APT is capable of concurrent method handling.
-
-<tag>message<item>
-A series of lines terminated by a blank line sent down one of the
-communication lines. The first line should have the form xxx TAG
-where xxx are digits forming the status code and TAG is an informational
-string
-
-<tag>acquire<item>
-The act of bring a URI into the local pathname space. This may simply
-be verifying the existence of the URI or actually downloading it from
-a remote site.
-
-</taglist>
-
-</sect>
-                                                                  <!-- }}} -->
-<chapt>Specification
-<!-- Overview		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Overview
-
-<p>
-All methods operate as a sub process of a main controlling parent. 3 FD's
-are opened for use by the method allowing two way communication and
-emergency error reporting. The FD's correspond to the well known unix FD's, 
-stdin, stdout and stderr.
-
-<p>
-Through operation of the method communication is done via http 
-style plain text. Specifically RFC-822 (like the Package file) fields
-are used to describe items and a numeric-like header is used to indicate
-what is happening. Each of these distinct communication messages should be
-sent quickly and without pause.
-
-<p> 
-In some instances APT may pre-invoke a method to allow things like file
-URI's to determine how many files are available locally.
-</sect>
-                                                                  <!-- }}} -->
-<!-- Message Overview	                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Message Overview
-
-<p>
-The first line of each message is called the message header. The first
-3 digits (called the Status Code) have the usual meaning found in the 
-http protocol. 1xx is informational, 2xx is successful and 4xx is failure. 
-The 6xx series is used to specify things sent to the method. After the 
-status code is an informational string provided for visual debugging.
-
-<list>
-<item>100 Capabilities - Method capabilities
-<item>101 Log - General Logging
-<item>102 Status - Inter-URI status reporting (login progress)
-<item>200 URI Start - URI is starting acquire
-<item>201 URI Done - URI is finished acquire
-<item>400 URI Failure - URI has failed to acquire
-<item>401 General Failure - Method did not like something sent to it
-<item>402 Authorization Required - Method requires authorization
-        to access the URI. Authorization is User/Pass
-<item>403 Media Failure - Method requires a media change	
-<item>600 URI Acquire - Request a URI be acquired
-<item>601 Configuration - Sends the configuration space
-<item>602 Authorization Credentials - Response to the 402 message
-<item>603 Media Changed - Response to the 403 message
-</list>
-
-Only the 6xx series of status codes is sent TO the method. Furthermore
-the method may not emit status codes in the 6xx range. The Codes 402
-and 403 require that the method continue reading all other 6xx codes
-until the proper 602/603 code is received. This means the method must be
-capable of handling an unlimited number of 600 messages.
-
-<p>
-The flow of messages starts with the method sending out a 
-<em>100 Capabilities</> and APT sending out a <em>601 Configuration</>.
-After that APT begins sending <em>600 URI Acquire</> and the method
-sends out <em>200 URI Start</>, <em>201 URI Done</> or 
-<em>400 URI Failure</>. No synchronization is performed, it is expected
-that APT will send <em>600 URI Acquire</> messages at -any- time and
-that the method should queue the messages. This allows methods like http
-to pipeline requests to the remote server. It should be noted however
-that APT will buffer messages so it is not necessary for the method
-to be constantly ready to receive them.
-</sect>
-                                                                  <!-- }}} -->
-<!-- Header Fields	                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Header Fields
-
-<p> 
-The following is a short index of the header fields that are supported
-
-<taglist>
-<tag>URI<item>URI being described by the message
-<tag>Filename<item>Location in the filesystem
-<tag>Last-Modified<item>A time stamp in RFC1123 notation for use by IMS checks
-<tag>IMS-Hit<item>The already existing item is valid
-<tag>Size<item>Size of the file in bytes
-<tag>Resume-Point<item>Location that transfer was started
-<tag>MD5-Hash<item>Computed MD5 hash for the file
-<tag>Message<item>String indicating some displayable message
-<tag>Media<item>String indicating the media name required
-<tag>Site<item>String indicating the site authorization is required for
-<tag>User<item>Username for authorization
-<tag>Password<item>Password for authorization
-<tag>Fail<item>Operation failed
-<tag>Drive<item>Drive the media should be placed in
-<tag>Config-Item<item>
-A string of the form <var>item</>=<var>value</> derived from the APT 
-configuration space. These may include method specific values and general
-values not related to the method. It is up to the method to filter out
-the ones it wants.
-<tag>Single-Instance<item>Requires that only one instance of the method be run
-                          This is a yes/no value.
-<tag>Pipeline<item>The method is capable of pipelining.
-<tag>Local<item>The method only returns Filename: fields.
-<tag>Send-Config<item>Send configuration to the method.
-<tag>Needs-Cleanup<item>The process is kept around while the files it returned
-are being used. This is primarily intended for CD-ROM and File URIs that need
-to unmount filesystems.
-<tag>Version<item>Version string for the method
-</taglist>
-
-This is a list of which headers each status code can use
-
-<taglist>
-<tag>100 Capabilities<item>
-Displays the capabilities of the method. Methods should set the
-pipeline bit if their underlying protocol supports pipelining. The
-only known method that does support pipelining is http.
-Fields: Version, Single-Instance, Pre-Scan, Pipeline, Send-Config, 
-Needs-Cleanup
-
-<tag>101 Log<item>
-A log message may be printed to the screen if debugging is enabled. This
-is only for debugging the method.
-Fields: Message
-
-<tag>102 Status<item>
-Message gives a progress indication for the method. It can be used to show
-pre-transfer status for Internet type methods.
-Fields: Message
-
-<tag>200 URI Start<item>
-Indicates the URI is starting to be transferred. The URI is specified
-along with stats about the file itself.
-Fields: URI, Size, Last-Modified, Resume-Point
-
-<tag>201 URI Done<item>
-Indicates that a URI has completed being transferred. It is possible
-to specify a <em>201 URI Done</> without a <em>URI Start</> which would
-mean no data was transferred but the file is now available. A Filename
-field is specified when the URI is directly available in the local 
-pathname space. APT will either directly use that file or copy it into 
-another location. It is possible to return Alt-* fields to indicate that
-another possibility for the URI has been found in the local pathname space.
-This is done if a decompressed version of a .gz file is found.
-Fields: URI, Size, Last-Modified, Filename, MD5-Hash
-
-<tag>400 URI Failure<item>
-Indicates a fatal URI failure. The URI is not retrievable from this source.
-As with <em>201 URI Done</> <em>200 URI Start</> is not required to precede
-this message
-Fields: URI, Message
-
-<tag>401 General Failure<item>
-Indicates that some unspecific failure has occurred and the method is unable
-to continue. The method should terminate after sending this message. It 
-is intended to check for invalid configuration options or other severe
-conditions.
-Fields: Message
-
-<tag>402 Authorization Required<item>
-The method requires a Username and Password pair to continue. After sending
-this message the method will expect APT to send a <em>602 Authorization 
-Credentials</> message with the required information. It is possible for
-a method to send this multiple times.
-Fields: Site
-
-<tag>403 Media Failure<item>
-A method that deals with multiple media requires that a new media be inserted.
-The Media field contains the name of the media to be inserted.
-Fields: Media, Drive
-
-<tag>600 URI Acquire<item>
-APT is requesting that a new URI be added to the acquire list. Last-Modified
-has the time stamp of the currently cache file if applicable. Filename
-is the name of the file that the acquired URI should be written to.
-Fields: URI, Filename Last-Modified
-
-<tag>601 Configuration<item>
-APT is sending the configuration space to the method. A series of
-Config-Item fields will be part of this message, each containing an entry
-from the configuration space.
-Fields: Config-Item.
-
-<tag>602 Authorization Credentials<item>
-This is sent in response to a <em>402 Authorization Required</> message. 
-It contains the entered username and password.
-Fields: Site, User, Password
-
-<tag>603 Media Changed<item>
-This is sent in response to a <em>403 Media Failure</> message. It
-indicates that the user has changed media and it is safe to proceed.
-Fields: Media, Fail
-</taglist>
-
-</sect>
-                                                                  <!-- }}} -->
-<!-- Method Notes		                                       {{{ -->
-<!-- ===================================================================== -->
-<sect>Notes
-
-<p>
-The methods supplied by the stock apt are:
-<enumlist>
-<item>cdrom - For Multi-Disc CD-ROMs
-<item>copy - (internal) For copying files around the filesystem
-<item>file - For local files
-<item>gzip - (internal) For decompression
-<item>http - For HTTP servers
-</enumlist>
-
-<p>
-The two internal methods, copy and gzip, are used by the acquire code to
-parallize and simplify the automatic decompression of package files as well 
-as copying package files around the file system. Both methods can be seen to 
-act the same except that one decompresses on the fly. APT uses them by
-generating a copy URI that is formed identically to a file URI. The destination
-file is send as normal. The method then takes the file specified by the 
-URI and writes it to the destination file. A typical set of operations may
-be:
-<example>
-http://foo.com/Packages.gz -> /bar/Packages.gz
-gzip:/bar/Packages.gz -> /bar/Packages.decomp
-rename Packages.decomp to /final/Packages
-</example>
-
-<p>
-The http method implements a fully featured HTTP/1.1 client that supports
-deep pipelining and reget. It works best when coupled with an apache 1.3
-server. The file method simply generates failures or success responses with
-the filename field set to the proper location. The cdrom method acts the same
-except that it checks that the mount point has a valid cdrom in it. It does 
-this by (effectively) computing a md5 hash of 'ls -l' on the mountpoint.
-
-</sect>
-                                                                  <!-- }}} -->
-
-</book>

doc/offline.sgml

@@ -1,74 +1,90 @@
-<!-- -*- mode: sgml; mode: fold -*- -->
-<!doctype debiandoc  PUBLIC  "-//DebianDoc//DTD DebianDoc//EN">
-<book>
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
+  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
+]>
+
+<book lang="en">
+
 <title>Using APT Offline</title>
 
-<author>Jason Gunthorpe <email>jgg@debian.org</email></author>
-<version>$Id: offline.sgml,v 1.8 2003/02/12 15:06:41 doogie Exp $</version>
+<bookinfo>
+
+<authorgroup>
+  <author>
+    <personname>Jason Gunthorpe</personname><email>jgg@debian.org</email>
+  </author>
+</authorgroup>
+
+<releaseinfo>Version &apt-product-version;</releaseinfo>
 
 <abstract>
-This document describes how to use APT in a non-networked environment, 
+<para>
+This document describes how to use APT in a non-networked environment,
 specifically a 'sneaker-net' approach for performing upgrades.
+</para>
 </abstract>
 
-<copyright>
-Copyright &copy; Jason Gunthorpe, 1999.
-<p>
+<copyright><year>1999</year><holder>Jason Gunthorpe</holder></copyright>
+
+<legalnotice>
+<title>License Notice</title>
+<para>
 "APT" and this document are free software; you can redistribute them and/or
-modify them under the terms of the GNU General Public License as published
-by the Free Software Foundation; either version 2 of the License, or (at your
+modify them under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or (at your
 option) any later version.
-
-<p>
+</para>
+<para>
 For more details, on Debian systems, see the file
 /usr/share/common-licenses/GPL for the full license.
-</copyright>
+</para>
+</legalnotice>
 
-<toc sect>
+</bookinfo>
 
-<chapt>Introduction
-<!-- Overview		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Overview
+<chapter id="ch1"><title>Introduction</title>
 
-<p>
+<section id="s1.1"><title>Overview</title>
+<para>
 Normally APT requires direct access to a Debian archive, either from a local
 media or through a network. Another common complaint is that a Debian machine
-is on a slow link, such as a modem and another machine has a very fast 
+is on a slow link, such as a modem and another machine has a very fast
 connection but they are physically distant.
-
-<p>
-The solution to this is to use large removable media such as a Zip disc or a 
+</para>
+<para>
+The solution to this is to use large removable media such as a Zip disc or a
 SuperDisk disc. These discs are not large enough to store the entire Debian
-archive but can easily fit a subset large enough for most users. The idea
-is to use APT to generate a list of packages that are required and then fetch
-them onto the disc using another machine with good connectivity. It is 
-even possible to use another Debian machine with APT or to use a completely 
-different OS and a download tool like wget. Let <em>remote host</em> mean the
-machine downloading the packages, and <em>target host</em> the one with bad or
-no connection.
-
-<p>
+archive but can easily fit a subset large enough for most users. The idea is
+to use APT to generate a list of packages that are required and then fetch them
+onto the disc using another machine with good connectivity. It is even
+possible to use another Debian machine with APT or to use a completely
+different OS and a download tool like wget. Let <emphasis>remote
+host</emphasis> mean the machine downloading the packages, and <emphasis>target
+host</emphasis> the one with bad or no connection.
+</para>
+<para>
 This is achieved by creatively manipulating the APT configuration file. The
 essential premise to tell APT to look on a disc for it's archive files. Note
 that the disc should be formated with a filesystem that can handle long file
 names such as ext2, fat32 or vfat.
+</para>
+</section>
 
-</sect>
-                                                                  <!-- }}} -->
+</chapter>
 
-<chapt>Using APT on both machines
-<!-- Overview		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Overview
+<chapter id="ch2"><title>Using APT on both machines</title>
 
-<p>
-APT being available on both machines gives the simplest configuration. The 
-basic idea is to place a copy of the status file on the disc and use the 
-remote machine to fetch the latest package files and decide which packages to 
+<section id="s2.1"><title>Overview</title>
+<para>
+APT being available on both machines gives the simplest configuration. The
+basic idea is to place a copy of the status file on the disc and use the remote
+machine to fetch the latest package files and decide which packages to
 download. The disk directory structure should look like:
-
-<example>
+</para>
+<screen>
   /disc/
     archives/
        partial/
@@ -77,36 +93,32 @@ download. The disk directory structure should look like:
     status
     sources.list
     apt.conf
-</example>
-
-</sect>
-                                                                  <!-- }}} -->
-<!-- The configuartion file                                            {{{ -->
-<!-- ===================================================================== -->
-<sect>The configuration file
-
-<p>
-The configuration file should tell APT to store its files on the disc and
-to use the configuration files on the disc as well. The sources.list should
-contain the proper sites that you wish to use from the remote machine, and
-the status file should be a copy of <em>/var/lib/dpkg/status</em> from the
-<em>target host</em>. Please note, if you are using a local archive you must use
-copy URIs, the syntax is identical to file URIs.
-
-<p>
-<em>apt.conf</em> must contain the necessary information to make APT use the 
-disc:
-
-<example>
+</screen>
+</section>
+
+<section id="s2.2"><title>The configuration file</title>
+<para>
+The configuration file should tell APT to store its files on the disc and to
+use the configuration files on the disc as well. The sources.list should
+contain the proper sites that you wish to use from the remote machine, and the
+status file should be a copy of <emphasis>/var/lib/dpkg/status</emphasis> from
+the <emphasis>target host</emphasis>. Please note, if you are using a local
+archive you must use copy URIs, the syntax is identical to file URIs.
+</para>
+<para>
+<emphasis>apt.conf</emphasis> must contain the necessary information to make
+APT use the disc:
+</para>
+<screen>
  APT
  {
    /* This is not necessary if the two machines are the same arch, it tells
       the remote APT what architecture the target machine is */
    Architecture "i386";
-   
+
    Get::Download-Only "true";
  };
- 
+
  Dir
  {
    /* Use the disc for state information and redirect the status file from
@@ -117,120 +129,120 @@ disc:
    // Binary caches will be stored locally
    Cache::archives "/disc/archives/";
    Cache "/tmp/";
-   
+
    // Location of the source list.
    Etc "/disc/";
- }; 
-</example>
-
+ };
+</screen>
+<para>
 More details can be seen by examining the apt.conf man page and the sample
-configuration file in <em>/usr/share/doc/apt/examples/apt.conf</em>.
-
-<p>
-On the target machine the first thing to do is mount the disc and copy 
-<em>/var/lib/dpkg/status</em> to it. You will also need to create the directories
-outlined in the Overview, <em>archives/partial/</em> and <em>lists/partial/</em>.
-Then take the disc to the remote machine and configure the sources.list. 
-On the remote machine execute the following:
-
-<example>
+configuration file in
+<emphasis>/usr/share/doc/apt/examples/apt.conf</emphasis>.
+</para>
+<para>
+On the target machine the first thing to do is mount the disc and copy
+<emphasis>/var/lib/dpkg/status</emphasis> to it. You will also need
+to create the directories outlined in the Overview,
+<emphasis>archives/partial/</emphasis> and
+<emphasis>lists/partial/</emphasis>. Then take the disc to the
+remote machine and configure the sources.list. On the remote
+machine execute the following:
+</para>
+<screen>
  # export APT_CONFIG="/disc/apt.conf"
  # apt-get update
  [ APT fetches the package files ]
  # apt-get dist-upgrade
  [ APT fetches all the packages needed to upgrade the target machine ]
-</example>
-
+</screen>
+<para>
 The dist-upgrade command can be replaced with any other standard APT commands,
-particularly dselect-upgrade. You can even use an APT front end such as 
-<em>dselect</em>. However this presents a problem in communicating your 
-selections back to the local computer.
-
-<p>
-Now the disc contains all of the index files and archives needed to upgrade
-the target machine. Take the disc back and run:
-
-<example>
+particularly dselect-upgrade. You can even use an APT front end such as
+<emphasis>dselect</emphasis>. However this presents a problem in communicating
+your selections back to the local computer.
+</para>
+<para>
+Now the disc contains all of the index files and archives needed to upgrade the
+target machine. Take the disc back and run:
+</para>
+<screen>
   # export APT_CONFIG="/disc/apt.conf"
   # apt-get check
   [ APT generates a local copy of the cache files ]
   # apt-get --no-d -o dir::state::status=/var/lib/dpkg/status dist-upgrade
   [ Or any other APT command ]
-</example>
-
-<p> 
-It is necessary for proper function to re-specify the status file to be the 
+</screen>
+<para>
+It is necessary for proper function to re-specify the status file to be the
 local one. This is very important!
-
-<p>
-If you are using dselect you can do the very risky operation of copying 
+</para>
+<para>
+If you are using dselect you can do the very risky operation of copying
 disc/status to /var/lib/dpkg/status so that any selections you made on the
-remote machine are updated. I highly recommend that people only make selections
-on the local machine - but this may not always be possible. DO NOT copy
-the status file if dpkg or APT have been run in the mean time!!
-
-</sect>
-                                                                  <!-- }}} -->
-
-<chapt>Using APT and wget
-<!-- Overview		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Overview
-
-<p>
-<em>wget</em> is a popular and portable download tool that can run on nearly
-any machine. Unlike the method above this requires that the Debian machine
-already has a list of available packages.
-
-<p>
+remote machine are updated. I highly recommend that people only make
+selections on the local machine - but this may not always be possible. DO NOT
+copy the status file if dpkg or APT have been run in the mean time!!
+</para>
+</section>
+
+</chapter>
+
+<chapter id="ch3"><title>Using APT and wget</title>
+
+<section id="s3.1"><title>Overview</title>
+<para>
+<emphasis>wget</emphasis> is a popular and portable download tool that can run
+on nearly any machine. Unlike the method above this requires that the Debian
+machine already has a list of available packages.
+</para>
+<para>
 The basic idea is to create a disc that has only the archive files downloaded
 from the remote site. This is done by using the --print-uris option to apt-get
 and then preparing a wget script to actually fetch the packages.
+</para>
+</section>
 
-</sect>
-                                                                  <!-- }}} -->
-<!-- Operation		                                               {{{ -->
-<!-- ===================================================================== -->
-<sect>Operation
-
-<p>
+<section id="s3.2"><title>Operation</title>
+<para>
 Unlike the previous technique no special configuration files are required. We
 merely use the standard APT commands to generate the file list.
-
-<example>
- # apt-get dist-upgrade 
+</para>
+<screen>
+ # apt-get dist-upgrade
  [ Press no when prompted, make sure you are happy with the actions ]
- # apt-get -qq --print-uris dist-upgrade > uris
- # awk '{print "wget -O " $2 " " $1}' < uris > /disc/wget-script
-</example>
-
-Any command other than dist-upgrade could be used here, including 
+ # apt-get -qq --print-uris dist-upgrade &gt; uris
+ # awk '{print "wget -O " $2 " " $1}' &lt; uris &gt; /disc/wget-script
+</screen>
+<para>
+Any command other than dist-upgrade could be used here, including
 dselect-upgrade.
-
-<p>
-The /disc/wget-script file will now contain a list of wget commands to execute 
+</para>
+<para>
+The /disc/wget-script file will now contain a list of wget commands to execute
 in order to fetch the necessary archives. This script should be run with the
-current directory as the disc's mount point so as to save the output on the 
+current directory as the disc's mount point so as to save the output on the
 disc.
-
-<p>
+</para>
+<para>
 The remote machine would do something like
-
-<example>
+</para>
+<screen>
   # cd /disc
   # sh -x ./wget-script
   [ wait.. ]
-</example>
-
+</screen>
+<para>
 Once the archives are downloaded and the disc returned to the Debian machine
 installation can proceed using,
-
-<example>
+</para>
+<screen>
   # apt-get -o dir::cache::archives="/disc/" dist-upgrade
-</example>
-
+</screen>
+<para>
 Which will use the already fetched archives on the disc.
+</para>
+</section>
+
+</chapter>
 
-</sect>
-                                                                  <!-- }}} -->
 </book>

doc/po4a.conf

@@ -27,18 +27,18 @@
 [type: manpage] apt-sortpkgs.1.xml $lang:$lang/apt-sortpkgs.$lang.1.xml add_$lang:xml.add
 [type: manpage] apt-ftparchive.1.xml $lang:$lang/apt-ftparchive.$lang.1.xml add_$lang:xml.add
 
-[type: sgml]    guide.sgml $lang:$lang/guide.$lang.sgml
-#                 add_$lang::$lang/addendum/debiandoc_$lang.add
-[type: sgml]    offline.sgml $lang:$lang/offline.$lang.sgml
-#                 add_$lang::$lang/addendum/debiandoc_$lang.add
-#[type: sgml]    cache.sgml $lang:$lang/cache.$lang.sgml \
-#                add_$lang::$lang/addendum/debiandoc_$lang.add
-#[type: sgml]    design.sgml $lang:$lang/design.$lang.sgml\
-#                add_$lang::$lang/addendum/debiandoc_$lang.add
-#[type: sgml]    dpkg-tech.sgml $lang:$lang/dpkg-tech.$lang.sgml\
-#                add_$lang::$lang/addendum/debiandoc_$lang.add
-#[type: sgml]    files.sgml $lang:$lang/files.$lang.sgml\
-#                add_$lang::$lang/addendum/debiandoc_$lang.add
-#[type: sgml]    method.sgml $lang:$lang/method.$lang.sgml\
-#                add_$lang::$lang/addendum/debiandoc_$lang.add
+[type: docbook] guide.dbk $lang:$lang/guide.$lang.dbk
+#                add_$lang::$lang/addendum/docbook_$lang.add
+[type: docbook] offline.dbk $lang:$lang/offline.$lang.dbk
+#                add_$lang::$lang/addendum/docbook_$lang.add
+#[type: docbook] cache.dbk $lang:$lang/cache.$lang.dbk \
+#                add_$lang::$lang/addendum/docbook_$lang.add
+#[type: docbook] design.dbk $lang:$lang/design.$lang.dbk\
+#                add_$lang::$lang/addendum/docbook_$lang.add
+#[type: docbook] dpkg-tech.dbk $lang:$lang/dpkg-tech.$lang.dbk\
+#                add_$lang::$lang/addendum/docbook_$lang.add
+#[type: docbook] files.dbk $lang:$lang/files.$lang.dbk\
+#                add_$lang::$lang/addendum/docbook_$lang.add
+#[type: docbook] method.dbk $lang:$lang/method.$lang.dbk\
+#                add_$lang::$lang/addendum/docbook_$lang.add
 

doc/sources.list.5.xml

@@ -1,15 +1,9 @@
 <?xml version="1.0" encoding="utf-8" standalone="no"?>
 <!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
   "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
-
-<!ENTITY % aptent SYSTEM "apt.ent">
-%aptent;
-
-<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent">
-%aptverbatiment;
-
-<!ENTITY % aptvendor SYSTEM "apt-vendor.ent">
-%aptvendor;
+<!ENTITY % aptent SYSTEM "apt.ent"> %aptent;
+<!ENTITY % aptverbatiment SYSTEM "apt-verbatim.ent"> %aptverbatiment;
+<!ENTITY % aptvendor SYSTEM "apt-vendor.ent"> %aptvendor;
 ]>
 
 <refentry>

ftparchive/apt-ftparchive.cc

@@ -781,33 +781,14 @@ static bool SimpleGenRelease(CommandLine &CmdL)
 }
 
 									/*}}}*/
-// Generate - Full generate, using a config file			/*{{{*/
+// DoGeneratePackagesAndSources - Helper for Generate                   /*{{{*/
 // ---------------------------------------------------------------------
-/* */
-static bool Generate(CommandLine &CmdL)
+static bool DoGeneratePackagesAndSources(Configuration &Setup,
+					 vector<PackageMap> &PkgList,
+					 struct CacheDB::Stats &SrcStats,
+					 struct CacheDB::Stats &Stats,
+					 CommandLine &CmdL)
 {
-   struct CacheDB::Stats SrcStats;
-   if (CmdL.FileSize() < 2)
-      return ShowHelp(CmdL);
-
-   struct timeval StartTime;
-   gettimeofday(&StartTime,0);   
-   struct CacheDB::Stats Stats;
-   
-   // Read the configuration file.
-   Configuration Setup;
-   if (ReadConfigFile(Setup,CmdL.FileList[1],true) == false)
-      return false;
-
-   vector<PackageMap> PkgList;
-   LoadTree(PkgList,Setup);
-   LoadBinDir(PkgList,Setup);
-
-   // Sort by cache DB to improve IO locality.
-   stable_sort(PkgList.begin(),PkgList.end(),PackageMap::DBCompare());
-   stable_sort(PkgList.begin(),PkgList.end(),PackageMap::SrcDBCompare());
-		
-   // Generate packages
    if (CmdL.FileSize() <= 2)
    {
       for (vector<PackageMap>::iterator I = PkgList.begin(); I != PkgList.end(); ++I)
@@ -876,9 +857,16 @@ static bool Generate(CommandLine &CmdL)
       if (I->TransWriter != NULL && I->TransWriter->DecreaseRefCounter() == 0)
 	 delete I->TransWriter;
 
-   if (_config->FindB("APT::FTPArchive::Contents",true) == false)
-      return true;
-   
+   return true;
+}
+
+                                                                        /*}}}*/
+// DoGenerateContents - Helper for Generate to generate the Contents    /*{{{*/
+// ---------------------------------------------------------------------
+static bool DoGenerateContents(Configuration &Setup,
+			       vector<PackageMap> &PkgList,
+			       CommandLine &CmdL)
+{
    c1out << "Packages done, Starting contents." << endl;
 
    // Sort the contents file list by date
@@ -935,17 +923,62 @@ static bool Generate(CommandLine &CmdL)
 	 break;
       }      
    }
+
+   return true;
+}
+
+									/*}}}*/
+// Generate - Full generate, using a config file			/*{{{*/
+// ---------------------------------------------------------------------
+/* */
+static bool Generate(CommandLine &CmdL)
+{
+   struct CacheDB::Stats SrcStats;
+   if (CmdL.FileSize() < 2)
+      return ShowHelp(CmdL);
+
+   struct timeval StartTime;
+   gettimeofday(&StartTime,0);
+   struct CacheDB::Stats Stats;
    
+   // Read the configuration file.
+   Configuration Setup;
+   if (ReadConfigFile(Setup,CmdL.FileList[1],true) == false)
+      return false;
+
+   vector<PackageMap> PkgList;
+   LoadTree(PkgList,Setup);
+   LoadBinDir(PkgList,Setup);
+
+   // Sort by cache DB to improve IO locality.
+   stable_sort(PkgList.begin(),PkgList.end(),PackageMap::DBCompare());
+   stable_sort(PkgList.begin(),PkgList.end(),PackageMap::SrcDBCompare());
+
+   // Generate packages
+   if (_config->FindB("APT::FTPArchive::ContentsOnly", false) == false)
+   {
+      if(DoGeneratePackagesAndSources(Setup, PkgList, SrcStats, Stats, CmdL) == false)
+         return false;
+   } else {
+      c1out << "Skipping Packages/Sources generation" << endl;
+   }
+
+   // do Contents if needed
+   if (_config->FindB("APT::FTPArchive::Contents", true) == true)
+      if (DoGenerateContents(Setup, PkgList, CmdL) == false)
+         return false;
+
    struct timeval NewTime;
-   gettimeofday(&NewTime,0);   
-   double Delta = NewTime.tv_sec - StartTime.tv_sec + 
+   gettimeofday(&NewTime,0);
+   double Delta = NewTime.tv_sec - StartTime.tv_sec +
                   (NewTime.tv_usec - StartTime.tv_usec)/1000000.0;
-   c1out << "Done. " << SizeToStr(Stats.Bytes) << "B in " << Stats.Packages 
+   c1out << "Done. " << SizeToStr(Stats.Bytes) << "B in " << Stats.Packages
          << " archives. Took " << TimeToStr((long)Delta) << endl;
-   
+
    return true;
 }
-									/*}}}*/
+
+                                                                        /*}}}*/
 // Clean - Clean out the databases					/*{{{*/
 // ---------------------------------------------------------------------
 /* */

methods/copy.cc

@@ -16,6 +16,7 @@
 #include <apt-pkg/acquire-method.h>
 #include <apt-pkg/error.h>
 #include <apt-pkg/hashes.h>
+#include <apt-pkg/configuration.h>
 
 #include <string>
 #include <sys/stat.h>
@@ -27,19 +28,32 @@
 class CopyMethod : public pkgAcqMethod
 {
    virtual bool Fetch(FetchItem *Itm);
+   void CalculateHashes(FetchResult &Res);
    
    public:
    
-   CopyMethod() : pkgAcqMethod("1.0",SingleInstance) {};
+   CopyMethod() : pkgAcqMethod("1.0",SingleInstance | SendConfig) {};
 };
 
+void CopyMethod::CalculateHashes(FetchResult &Res)
+{
+   Hashes Hash;
+   FileFd::CompressMode CompressMode = FileFd::None;
+   if (_config->FindB("Acquire::GzipIndexes", false) == true)
+      CompressMode = FileFd::Extension;
+
+   FileFd Fd(Res.Filename, FileFd::ReadOnly, CompressMode);
+   Hash.AddFD(Fd);
+   Res.TakeHashes(Hash);
+}
+
 // CopyMethod::Fetch - Fetch a file					/*{{{*/
 // ---------------------------------------------------------------------
 /* */
 bool CopyMethod::Fetch(FetchItem *Itm)
 {
-   URI Get = Itm->Uri;
-   std::string File = Get.Path;
+   // this ensures that relative paths work in copy
+   std::string File = Itm->Uri.substr(Itm->Uri.find(':')+1);
 
    // Stat the file and send a start message
    struct stat Buf;
@@ -54,6 +68,14 @@ bool CopyMethod::Fetch(FetchItem *Itm)
    Res.IMSHit = false;      
    URIStart(Res);
    
+   // just calc the hashes if the source and destination are identical
+   if (File == Itm->DestFile)
+   {
+      CalculateHashes(Res);
+      URIDone(Res);
+      return true;
+   }
+
    // See if the file exists
    FileFd From(File,FileFd::ReadOnly);
    FileFd To(Itm->DestFile,FileFd::WriteAtomic);
@@ -82,10 +104,7 @@ bool CopyMethod::Fetch(FetchItem *Itm)
    if (utimes(Res.Filename.c_str(), times) != 0)
       return _error->Errno("utimes",_("Failed to set modification time"));
 
-   Hashes Hash;
-   FileFd Fd(Res.Filename, FileFd::ReadOnly);
-   Hash.AddFD(Fd);
-   Res.TakeHashes(Hash);
+   CalculateHashes(Res);
 
    URIDone(Res);
    return true;

methods/http.cc

@@ -34,6 +34,7 @@
 #include <apt-pkg/hashes.h>
 #include <apt-pkg/netrc.h>
 #include <apt-pkg/strutl.h>
+#include <apt-pkg/proxy.h>
 
 #include <stddef.h>
 #include <stdlib.h>
@@ -304,6 +305,7 @@ bool HttpServerState::Open()
    Persistent = true;
    
    // Determine the proxy setting
+   AutoDetectProxy(ServerName);
    string SpecificProxy = _config->Find("Acquire::http::Proxy::" + ServerName.Host);
    if (!SpecificProxy.empty())
    {
@@ -744,7 +746,7 @@ void HttpMethod::SendReq(FetchItem *Itm)
    Req << "\r\n";
 
    if (Debug == true)
-      cerr << Req << endl;
+      cerr << Req.str() << endl;
 
    Server->WriteResponse(Req.str());
 }
@@ -762,66 +764,6 @@ bool HttpMethod::Configuration(string Message)
 				  PipelineDepth);
    Debug = _config->FindB("Debug::Acquire::http",false);
 
-   // Get the proxy to use
-   AutoDetectProxy();
-
-   return true;
-}
-									/*}}}*/
-// HttpMethod::AutoDetectProxy - auto detect proxy			/*{{{*/
-// ---------------------------------------------------------------------
-/* */
-bool HttpMethod::AutoDetectProxy()
-{
-   // option is "Acquire::http::Proxy-Auto-Detect" but we allow the old
-   // name without the dash ("-")
-   AutoDetectProxyCmd = _config->Find("Acquire::http::Proxy-Auto-Detect",
-                                      _config->Find("Acquire::http::ProxyAutoDetect"));
-
-   if (AutoDetectProxyCmd.empty())
-      return true;
-
-   if (Debug)
-      clog << "Using auto proxy detect command: " << AutoDetectProxyCmd << endl;
-
-   int Pipes[2] = {-1,-1};
-   if (pipe(Pipes) != 0)
-      return _error->Errno("pipe", "Failed to create Pipe");
-
-   pid_t Process = ExecFork();
-   if (Process == 0)
-   {
-      close(Pipes[0]);
-      dup2(Pipes[1],STDOUT_FILENO);
-      SetCloseExec(STDOUT_FILENO,false);
-
-      const char *Args[2];
-      Args[0] = AutoDetectProxyCmd.c_str();
-      Args[1] = 0;
-      execv(Args[0],(char **)Args);
-      cerr << "Failed to exec method " << Args[0] << endl;
-      _exit(100);
-   }
-   char buf[512];
-   int InFd = Pipes[0];
-   close(Pipes[1]);
-   int res = read(InFd, buf, sizeof(buf)-1);
-   ExecWait(Process, "ProxyAutoDetect", true);
-
-   if (res < 0)
-      return _error->Errno("read", "Failed to read");
-   if (res == 0)
-      return _error->Warning("ProxyAutoDetect returned no data");
-
-   // add trailing \0
-   buf[res] = 0;
-
-   if (Debug)
-      clog << "auto detect command returned: '" << buf << "'" << endl;
-
-   if (strstr(buf, "http://") == buf)
-      _config->Set("Acquire::http::proxy", _strstrip(buf));
-
    return true;
 }
 									/*}}}*/

methods/http.h

@@ -124,9 +124,6 @@ class HttpMethod : public ServerMethod
    public:
    virtual void SendReq(FetchItem *Itm);
 
-   /** \brief Try to AutoDetect the proxy */
-   bool AutoDetectProxy();
-
    virtual bool Configuration(std::string Message);
 
    virtual ServerState * CreateServerState(URI uri);

methods/https.cc

@@ -20,6 +20,7 @@
 #include <apt-pkg/configuration.h>
 #include <apt-pkg/macros.h>
 #include <apt-pkg/strutl.h>
+#include <apt-pkg/proxy.h>
 
 #include <sys/stat.h>
 #include <sys/time.h>
@@ -107,6 +108,9 @@ void HttpsMethod::SetupProxy()  					/*{{{*/
 {
    URI ServerName = Queue->Uri;
 
+   // Determine the proxy setting
+   AutoDetectProxy(ServerName);
+
    // Curl should never read proxy settings from the environment, as
    // we determine which proxy to use.  Do this for consistency among
    // methods and prevent an environment variable overriding a

methods/rsh.cc

@@ -218,17 +218,20 @@ bool RSHConn::WriteMsg(std::string &Text,bool Sync,const char *Fmt,...)
    va_list args;
    va_start(args,Fmt);
 
-   // sprintf the description
-   char S[512];
-   vsnprintf(S,sizeof(S) - 4,Fmt,args);
+   // sprintf into a buffer
+   char Tmp[1024];
+   vsnprintf(Tmp,sizeof(Tmp),Fmt,args);
    va_end(args);
 
+   // concat to create the real msg
+   std::string Msg;
    if (Sync == true)
-      strcat(S," 2> /dev/null || echo\n");
+      Msg = std::string(Tmp) + " 2> /dev/null || echo\n";
    else
-      strcat(S," 2> /dev/null\n");
+      Msg = std::string(Tmp) + " 2> /dev/null\n";
 
    // Send it off
+   const char *S = Msg.c_str();
    unsigned long Len = strlen(S);
    unsigned long Start = 0;
    while (Len != 0)

methods/server.cc

@@ -44,7 +44,8 @@ time_t ServerMethod::FailTime = 0;
 // ---------------------------------------------------------------------
 /* Returns 0 if things are OK, 1 if an IO error occurred and 2 if a header
    parse error occurred */
-ServerState::RunHeadersResult ServerState::RunHeaders(FileFd * const File)
+ServerState::RunHeadersResult ServerState::RunHeaders(FileFd * const File,
+                                                      const std::string &Uri)
 {
    State = Header;
    
@@ -66,7 +67,7 @@ ServerState::RunHeadersResult ServerState::RunHeaders(FileFd * const File)
 	 continue;
 
       if (Owner->Debug == true)
-	 clog << Data;
+	 clog << "Answer for: " << Uri << endl << Data;
       
       for (string::const_iterator I = Data.begin(); I < Data.end(); ++I)
       {
@@ -478,7 +479,7 @@ int ServerMethod::Loop()
       Fetch(0);
       
       // Fetch the next URL header data from the server.
-      switch (Server->RunHeaders(File))
+      switch (Server->RunHeaders(File, Queue->Uri))
       {
 	 case ServerState::RUN_HEADERS_OK:
 	 break;

methods/server.h

@@ -68,7 +68,7 @@ struct ServerState
       RUN_HEADERS_PARSE_ERROR
    };
    /** \brief Get the headers before the data */
-   RunHeadersResult RunHeaders(FileFd * const File);
+   RunHeadersResult RunHeaders(FileFd * const File, const std::string &Uri);
 
    bool Comp(URI Other) const {return Other.Host == ServerName.Host && Other.Port == ServerName.Port;};
    virtual void Reset() {Major = 0; Minor = 0; Result = 0; Code[0] = '\0'; Size = 0;