Represents the process by which a pkgAcquire object should retrieve a file or a collection of files. More...

#include <apt-pkg/acquire-item.h>

Inheritance diagram for pkgAcquire:

Collaboration diagram for pkgAcquire:

Classes
struct	QItem
	A single item placed in this queue. More...

Public Types
enum	ItemState { StatIdle , StatFetching , StatDone , StatError , StatAuthError , StatTransientNetworkError }
	The current status of this item. More...

enum	RunResult { Continue , Failed , Cancelled }
	Provides information on how a download terminated. More...

typedef std::vector< Item * >::iterator	ItemIterator

typedef std::vector< Item * >::const_iterator	ItemCIterator

Public Member Functions
virtual void	Failed (std::string const &Message, pkgAcquire::MethodConfig const *const Cnf)
	Invoked by the acquire worker when the object couldn't be fetched.

APT_HIDDEN void	FailMessage (std::string const &Message)

virtual bool	VerifyDone (std::string const &Message, pkgAcquire::MethodConfig const *const Cnf)
	Invoked by the acquire worker to check if the successfully fetched object is also the objected we wanted to have.

virtual void	Done (std::string const &Message, HashStringList const &Hashes, pkgAcquire::MethodConfig const *const Cnf)
	Invoked by the acquire worker when the object was fetched successfully.

virtual void	Start (std::string const &Message, unsigned long long const Size)
	Invoked when the worker starts to fetch this object.

virtual std::string	Custom600Headers () const
	Custom headers to be sent to the fetch process.

APT_HIDDEN std::unordered_map< std::string, std::string > &	ModifyCustomFields ()

APT_HIDDEN bool	PopAlternativeURI (std::string &NewURI)

APT_HIDDEN bool	IsGoodAlternativeURI (std::string const &AltUri) const

APT_HIDDEN void	PushAlternativeURI (std::string &&NewURI, std::unordered_map< std::string, std::string > &&fields, bool const at_the_back)

APT_HIDDEN void	RemoveAlternativeSite (std::string const &AltUri)

virtual std::string	DescURI () const =0
	A "descriptive" URI-like string.

virtual std::string	ShortDesc () const
	Short item description.

virtual void	Finished ()
	Invoked by the worker when the download is completely done.

virtual HashStringList	GetExpectedHashes () const =0

std::string	HashSum () const

virtual bool	HashesRequired () const

pkgAcquire *	GetOwner () const

pkgAcquire::ItemDesc &	GetItemDesc ()

virtual bool	IsTrusted () const

void	SetActiveSubprocess (std::string const &subprocess)
	Set the name of the current active subprocess.

	Item (pkgAcquire *const Owner)
	Initialize an item.

virtual	~Item ()
	Remove this item from its owner's queue by invoking pkgAcquire::Remove.

bool APT_HIDDEN	IsRedirectionLoop (std::string const &NewURI)

int APT_HIDDEN	Priority ()
	The priority of the item, used for queuing.

void APT_HIDDEN	FetchAfter (time_point FetchAfter)
	internal clock definitions to avoid typing all that all over the place

time_point APT_HIDDEN	FetchAfter ()

bool	QueueItem (pkgAcquire::Queue::QItem *Item)
	Tell the subprocess to download the given item.

APT_HIDDEN bool	ReplyAux (pkgAcquire::ItemDesc const &Item)

bool	Start ()
	Start up the worker and fill in Config.

void	Pulse ()
	Update the worker statistics (CurrentSize, TotalSize, etc).

const MethodConfig *	GetConf () const

	Worker (Queue OwnerQ, MethodConfig Config, pkgAcquireStatus *Log)
	Create a new Worker to download files.

	Worker (MethodConfig *Config)
	Create a new Worker that should just retrieve information about the fetch method.

virtual	~Worker ()
	Clean up this worker.

MethodConfig *	GetConfig (std::string Access)
	Retrieve information about a fetch method by name.

RunResult	Run (int PulseInterval=500000)
	Download all the items that have been Add()ed to this download process.

void	Shutdown ()
	Remove all items from this download process, terminate all download workers, and empty all queues.

Worker *	WorkersBegin ()
	Get the first Worker object.

Worker *	WorkerStep (Worker *I) APT_PURE
	Advance to the next Worker object.

ItemIterator	ItemsBegin ()
	Get the head of the list of items.

ItemCIterator	ItemsBegin () const

ItemIterator	ItemsEnd ()
	Get the end iterator of the list of items.

ItemCIterator	ItemsEnd () const

UriIterator	UriBegin ()
	Get the head of the list of enqueued item URIs.

UriIterator	UriEnd ()
	Get the end iterator of the list of enqueued item URIs.

bool	Clean (std::string Dir)

bool	CleanLists (std::string const &Dir)

unsigned long long	TotalNeeded ()

unsigned long long	FetchNeeded ()

unsigned long long	PartialPresent ()

void	SetLog (pkgAcquireStatus *Progress)

bool	GetLock (std::string const &Lock)
	acquire lock and perform directory setup

	pkgAcquire (pkgAcquireStatus *Log)
	Construct a new pkgAcquire.

virtual	~pkgAcquire ()
	Destroy this pkgAcquire object.

bool	Enqueue (ItemDesc &Item)
	Insert the given fetch request into this queue.

bool	Dequeue (Item *Owner)
	Remove all fetch requests for the given item from this queue.

QItem *	FindItem (std::string URI, pkgAcquire::Worker *Owner) APT_PURE
	Locate an item in this queue.

bool	ItemStart (QItem *Itm, unsigned long long Size)

bool	ItemDone (QItem *Itm)
	Remove the given item from this queue and set its state to pkgAcquire::Item::StatDone.

bool	Startup ()
	Start the worker process associated with this queue.

bool	Shutdown (bool Final)
	Shut down the worker process associated with this queue.

bool	Cycle ()
	Send idle items to the worker process.

void	Bump ()
	Check for items that could be enqueued.

	Queue (std::string const &Name, pkgAcquire *const Owner)
	Create a new Queue.

virtual	~Queue ()

void	operator++ ()

void	operator++ (int)

pkgAcquire::Queue::QItem const *	operator-> () const

bool	operator!= (UriIterator const &rhs) const

bool	operator== (UriIterator const &rhs) const

	UriIterator (pkgAcquire::Queue *Q)
	Create a new UriIterator.

	MethodConfig ()
	Set up the default method parameters.

APT_HIDDEN bool	GetAuxRequests () const

APT_HIDDEN void	SetAuxRequests (bool const value)

APT_HIDDEN bool	GetSendURIEncoded () const

APT_HIDDEN void	SetSendURIEncoded (bool const value)

Static Public Member Functions
static APT_HIDDEN std::string	URIEncode (std::string const &part)

Public Attributes
enum pkgAcquire::ItemState	Status

std::string	ErrorText
	Contains a textual description of the error encountered if ItemState is StatError or StatAuthError.

unsigned long long	FileSize
	The size of the object to fetch.

unsigned long long	PartialSize
	How much of the object was already fetched.

std::string	ActiveSubprocess
	contains the name of the subprocess that is operating on this object (for instance, "gzip", "rred" or "gpgv"). This is obsoleting #Mode from above as it can manage the lifetime of included string properly.

unsigned long	ID
	A client-supplied unique identifier.

bool	Complete
	If true, the entire object has been successfully fetched.

bool	Local
	If true, the URI of this object is "local".

std::string	UsedMirror

unsigned int	QueueCounter
	The number of fetch queues into which this item has been inserted.

unsigned int	ExpectedAdditionalItems
	The number of additional fetch items that are expected once this item is done.

std::string	DestFile
	The name of the file into which the retrieved object will be written.

unsigned int	Retries
	Number of retries.

pkgAcquire::Queue::QItem *	CurrentItem
	The queue entry that is currently being downloaded.

std::string	Status
	The most recent status string received from the subprocess.

std::string	URI
	URI from which to download this item.

std::string	Description
	description of this item.

std::string	ShortDesc
	shorter description of this item.

Item *	Owner
	underlying item which is to be downloaded.

MethodConfig *	Next
	The next link on the acquire method list.

std::string	Version
	The implementation version of this acquire method.

bool	SingleInstance
	If true, only one download queue should be created for this method.

bool	Pipeline
	If true, this method supports pipelined downloading.

bool	SendConfig
	If true, the worker process should send the entire APT configuration tree to the fetch subprocess when it starts up.

bool	LocalOnly
	If true, this fetch method does not require network access; all files are to be acquired from the local disk.

bool	NeedsCleanup
	If true, the subprocess has to carry out some cleanup actions before shutting down.

bool	Removable
	If true, this fetch method acquires files from removable media.

Protected Types
enum	RenameOnErrorState { HashSumMismatch , SizeMismatch , InvalidFormat , SignatureError , NotClearsigned , MaximumSizeExceeded , PDiffError }

enum	QueueStrategy { QueueHost , QueueAccess }
	Represents the queuing strategy for remote URIs. More...

Protected Member Functions
bool	RenameOnError (RenameOnErrorState const state)
	Rename failed file and set error.

virtual bool	QueueURI (ItemDesc &Item)
	Insert this item into its owner's queue.

void	Dequeue ()
	Remove this item from its owner's queue.

bool	Rename (std::string const &From, std::string const &To)
	Rename a file without modifying its timestamp.

virtual std::string	GetFinalFilename () const
	Get the full pathname of the final file for the current URI.

void	Construct ()
	Common code for the constructor.

bool	ReadMessages ()
	Retrieve any available messages from the subprocess.

bool	RunMessages ()
	Parse and dispatch pending messages.

bool	InFdReady ()
	Read and dispatch any pending messages from the subprocess.

bool	OutFdReady ()
	Send any pending commands to the subprocess.

bool	Capabilities (std::string Message)
	Handle a 100 Capabilities response from the subprocess.

bool	SendConfiguration ()
	Send a 601 Configuration message (containing the APT configuration) to the subprocess.

bool	MediaChange (std::string Message)
	Handle a 403 Media Change message.

bool	MethodFailure ()
	Invoked when the worked process dies unexpectedly.

void	ItemDone ()
	Invoked when a fetch job is completed, either successfully or unsuccessfully.

void	Add (Item *Item)
	Add the given item to the list of items.

void	Remove (Item *Item)
	Remove the given item from the list of items.

void	Add (Worker *Work)
	Add the given worker to the list of workers.

void	Remove (Worker *Work)
	Remove the given worker from the list of workers.

void	Enqueue (ItemDesc &Item)
	Insert the given fetch request into the appropriate queue.

void	Dequeue (Item *Item)
	Remove all fetch requests for this item from all queues.

std::string	QueueName (std::string URI, MethodConfig const *&Config)
	Determine the fetch method and queue of a URI.

virtual void	SetFds (int &Fd, fd_set RSet, fd_set WSet)
	Build up the set of file descriptors upon which select() should block.

virtual bool	RunFds (fd_set RSet, fd_set WSet)

void	Bump ()
	Check for idle queues with ready-to-fetch items.

Protected Attributes
pkgAcquire *const	Owner
	The acquire object with which this item is associated.

pkgAcquire::ItemDesc	Desc
	The item that is currently being downloaded.

Worker *	NextQueue
	The next link on the Queue list.

Worker *	NextAcquire
	The next link on the Acquire list.

Queue *	OwnerQ
	The Queue with which this worker is associated.

pkgAcquireStatus *	Log
	The download progress indicator to which progress messages should be sent.

MethodConfig *	Config
	The configuration of this method. On startup, the target of this pointer is filled in with basic data about the method, as reported by the worker.

std::string	Access
	The access method to be used by this worker.

pid_t	Process
	The PID of the subprocess.

int	InFd
	A file descriptor connected to the standard output of the subprocess.

int	OutFd
	A file descriptor connected to the standard input of the subprocess.

int	PrivSepSocketFd
	The socket to send SCM_RIGHTS message through.

int	PrivSepSocketFdChild

bool	InReady
	Set to true if the worker is in a state in which it might generate data or command responses.

bool	OutReady
	Set to true if the worker is in a state in which it is legal to send commands to it.

bool	Debug

std::vector< std::string >	MessageQueue
	The raw text values of messages received from the worker, in sequence.

std::string	OutQueue
	Buffers pending writes to the subprocess.

std::vector< Item * >	Items
	A list of items to download.

Queue *	Queues
	The head of the list of active queues.

Worker *	Workers
	The head of the list of active workers.

MethodConfig *	Configs
	The head of the list of acquire method configurations.

unsigned long	ToFetch
	The number of files which are to be fetched.

enum pkgAcquire::QueueStrategy	QueueMode

bool const	Debug
	If true, debugging information will be dumped to std::clog.

bool	Running
	If true, a download is currently in progress.

std::string	Name
	The name of this queue.

QItem *	Items
	The head of the list of items contained in this queue.

pkgAcquire::Worker *	Workers
	The head of the list of workers associated with this queue.

pkgAcquire *	Owner
	the download scheduler with which this queue is associated.

signed long	PipeDepth
	The number of entries in this queue that are currently being downloaded.

unsigned long	MaxPipeDepth
	The maximum number of entries that this queue will attempt to download at once.

Friends
class	pkgAcqMetaBase

class	pkgAcqMetaClearSig

class	Queue

class	Item

class	pkgAcquire::UriIterator

class	pkgAcquire::Worker

Detailed Description

Represents the process by which a pkgAcquire object should retrieve a file or a collection of files.

Information about the properties of a single acquire method. {{{.

Iterates over all the URIs being fetched by a pkgAcquire object. {{{.

A single download queue in a pkgAcquire object. {{{.

Represents a single download source from which an item should be downloaded.

The core download scheduler. {{{.

A fetch subprocess.

By convention, Item subclasses should insert themselves into the acquire queue when they are created by calling QueueURI(), and remove themselves by calling Dequeue() when either Done() or Failed() is invoked. Item objects are also responsible for notifying the download progress indicator (accessible via Owner->Log) of their status.

See also: pkgAcquire

A worker process is responsible for one stage of the fetch. This class encapsulates the communications protocol between the master process and the worker, from the master end.

Each worker is intrinsically placed on two linked lists. The Queue list (maintained in the NextQueue variable) is maintained by the pkgAcquire::Queue class; it represents the set of workers assigned to a particular queue. The Acquire list (maintained in the NextAcquire variable) is maintained by the pkgAcquire class; it represents the set of active workers for a particular pkgAcquire object.

Todo:: Like everything else in the Acquire system, this has way too many protected items.

See also: pkgAcqMethod, pkgAcquire::Item, pkgAcquire

This class represents an ongoing download. It manages the lists of active and pending downloads and handles setting up and tearing down download-related structures.

Todo:: Why all the protected data items and methods?

An item may have several associated ItemDescs over its lifetime.

Todo:: Why so many protected values?

Member Enumeration Documentation

◆ ItemState

enum ItemState

The current status of this item.

Enumerator
StatIdle	The item is waiting to be downloaded.
StatFetching	The item is currently being downloaded.
StatDone	The item has been successfully downloaded.
StatError	An error was encountered while downloading this item.
StatAuthError	The item was downloaded but its authenticity could not be verified.
StatTransientNetworkError	The item was could not be downloaded because of a transient network error (e.g. network down)

◆ QueueStrategy

enum QueueStrategy

protected

Represents the queuing strategy for remote URIs.

Enumerator
QueueHost	Generate one queue for each protocol/host combination; downloads from multiple hosts can proceed in parallel.
QueueAccess	Generate a single queue for each protocol; serialize downloads from multiple hosts.

◆ RunResult

enum RunResult

Provides information on how a download terminated.

Enumerator
Continue	All files were fetched successfully.
Failed	Some files failed to download.
Cancelled	The download was cancelled by the user (i.e., Log's pkgAcquireStatus::Pulse() method returned false).

Constructor & Destructor Documentation

◆ ~Worker()

virtual ~Worker ( )

virtual

Clean up this worker.

Closes the file descriptors; if MethodConfig::NeedsCleanup is false, also rudely interrupts the worker with a SIGINT.

◆ ~pkgAcquire()

~pkgAcquire ( )

virtual

Destroy this pkgAcquire object.

Destroys all queue, method, and item objects associated with this download.

References Configs, MethodConfig(), and Shutdown().

◆ ~Queue()

virtual ~Queue ( )

virtual

Shut down all the worker processes associated with this queue and empty the queue.

Member Function Documentation

◆ Bump() [1/2]

void Bump ( )

protected

Check for idle queues with ready-to-fetch items.

Called by pkgAcquire::Queue::Done each time an item is dequeued but remains on some queues; i.e., another queue should start fetching it.

References Queues.

◆ Bump() [2/2]

void Bump ( )

Check for items that could be enqueued.

Call this after an item placed in multiple queues has gone from the pkgAcquire::Item::StatFetching state to the pkgAcquire::Item::StatIdle state, to possibly refill an empty queue. This is an alias for Cycle().

Todo:: Why both this and Cycle()? Are they expected to be different someday?

◆ Capabilities()

bool Capabilities ( std::string Message )

protected

Handle a 100 Capabilities response from the subprocess.

Parameters

Message the raw text of the message from the subprocess.

The message will be parsed and its contents used to fill Config. If Config is NULL, this routine is a NOP.

Returns: true.

◆ Clean()

bool Clean ( std::string Dir )

Deletes each entry in the given directory that is not being downloaded by this object. For instance, when downloading new list files, calling Clean() will delete the old ones.

Parameters

Dir	The directory to be cleaned out.

Returns: true if the directory exists and is readable.

References Items.

◆ CleanLists()

bool CleanLists ( std::string const & Dir )

Deletes each package list and index file in the given directory.

Parameters

Dir	The directory to be cleaned.

Returns: true if the directory exists and is readable.

◆ Construct()

void Construct ( )

protected

Common code for the constructor.

Initializes NextQueue and NextAcquire to NULL; Process, InFd, and OutFd to -1, OutReady and InReady to false, and Debug from _config.

◆ Custom600Headers()

virtual std::string Custom600Headers ( ) const

virtual

Custom headers to be sent to the fetch process.

Returns: a string containing RFC822-style headers that are to be inserted into the 600 URI Acquire message sent to the fetch subprocess. The headers are inserted after a newline-less line, so they should (if nonempty) have a leading newline and no trailing newline.

◆ Cycle()

bool Cycle ( )

Send idle items to the worker process.

Fills up the pipeline by inserting idle items into the worker's queue.

◆ Dequeue()

bool Dequeue ( Item * Owner )

Remove all fetch requests for the given item from this queue.

Returns: true if at least one request was removed from the queue.

◆ DescURI()

virtual std::string DescURI ( ) const

pure virtual

A "descriptive" URI-like string.

Returns: a URI that should be used to describe what is being fetched.

◆ Done()

virtual void Done	(	std::string const &	Message,
		HashStringList const &	Hashes,
		pkgAcquire::MethodConfig const *const	Cnf
	)

virtual

Invoked by the acquire worker when the object was fetched successfully.

Note that the object might not have been written to DestFile; check for the presence of an Alt-Filename entry in Message to find the file to which it was really written.

Done is often used to switch from one stage of the processing to the next (e.g. fetching, unpacking, copying). It is one branch of the continuation of the fetch process.

Parameters

Message	Data from the acquire method. Use LookupTag() to parse it.
Hashes	The HashSums of the object that was fetched.
Cnf	The method via which the object was fetched.

See also: pkgAcqMethod

◆ Enqueue() [1/2]

void Enqueue ( ItemDesc & Item )

protected

Insert the given fetch request into the appropriate queue.

Parameters

Item	The URI to download and the item to download it for. Copied by value into the queue; no reference to Item is retained.

References Config, Debug, Log, MethodConfig(), Name, QueueName(), Queues, Running, and ToFetch.

◆ Enqueue() [2/2]

bool Enqueue ( ItemDesc & Item )

Insert the given fetch request into this queue.

Returns: true if the queuing was successful. May return false if the Item is already in the queue

◆ Failed()

virtual void Failed	(	std::string const &	Message,
		pkgAcquire::MethodConfig const *const	Cnf
	)

virtual

Invoked by the acquire worker when the object couldn't be fetched.

This is a branch of the continuation of the fetch process.

Parameters

Message	An RFC822-formatted message from the acquire method describing what went wrong. Use LookupTag() to parse it.
Cnf	The method via which the worker tried to fetch this object.

See also: pkgAcqMethod

◆ FetchNeeded()

APT_PURE unsigned long long FetchNeeded ( )

Returns: the size in bytes of all non-local items included in this download.

References ItemsBegin(), and ItemsEnd().

◆ FindItem()

QItem * FindItem	(	std::string	URI,
		pkgAcquire::Worker *	Owner
	)

Locate an item in this queue.

Parameters

URI	A URI to match against.
Owner	A pkgAcquire::Worker to match against.

Returns: the first item in the queue whose URI is URI and that is being downloaded by Owner.

◆ GetConf()

const MethodConfig * GetConf ( ) const

inline

Returns: The fetch method configuration.

◆ GetConfig()

pkgAcquire::MethodConfig * GetConfig ( std::string Access )

Retrieve information about a fetch method by name.

Parameters

Access The name of the method to look up.

Returns: the method whose name is Access, or NULL if no such method exists.

References Access, Configs, MethodConfig(), and Worker().

Referenced by QueueName().

◆ GetExpectedHashes()

virtual HashStringList GetExpectedHashes ( ) const

pure virtual

Returns: HashSums the DestFile is supposed to have in this stage

◆ GetLock()

bool GetLock ( std::string const & Lock )

acquire lock and perform directory setup

Parameters

Lock	defines a lock file that should be acquired to ensure only one Acquire class is in action at the time or an empty string if no lock file should be used. If set also all needed directories will be created and setup.

References GetLock().

Referenced by GetLock().

◆ GetOwner()

pkgAcquire * GetOwner ( ) const

Returns: the acquire process with which this item is associated.

◆ HashesRequired()

virtual bool HashesRequired ( ) const

inlinevirtual

Returns: if having no hashes is a hard failure or not

Idealy this is always true for every subclass, but thanks to historical grow we don't have hashes for all files in all cases in all steps, so it is slightly more complicated than it should be.

◆ HashSum()

std::string HashSum ( ) const

Returns: the 'best' hash for display proposes like –print-uris

◆ InFdReady()

bool InFdReady ( )

protected

Read and dispatch any pending messages from the subprocess.

Returns: false if the subprocess died unexpectedly while a message was being transmitted.

◆ IsTrusted()

virtual bool IsTrusted ( ) const

virtual

Returns: true if this object is being fetched from a trusted source.

◆ Item()

Item ( pkgAcquire *const Owner )

explicit

Initialize an item.

Adds the item to the list of items known to the acquire process, but does not place it into any fetch queues (you must manually invoke QueueURI() to do so).

Parameters

Owner The new owner of this item.

◆ ItemDone() [1/2]

void ItemDone ( )

protected

Invoked when a fetch job is completed, either successfully or unsuccessfully.

Resets the status information for the worker process.

◆ ItemDone() [2/2]

bool ItemDone ( QItem * Itm )

Remove the given item from this queue and set its state to pkgAcquire::Item::StatDone.

If this is the only queue containing the item, the item is also removed from the main queue by calling pkgAcquire::Dequeue.

Parameters

Itm	The item to remove.

Returns: true if no errors are encountered.

◆ ItemStart()

bool ItemStart	(	QItem *	Itm,
		unsigned long long	Size
	)

Presumably this should start downloading an item?

Todo:: Unimplemented. Implement it or remove?

◆ MediaChange()

bool MediaChange ( std::string Message )

protected

Handle a 403 Media Change message.

Parameters

Message the raw text of the message; the Media field indicates what type of media should be changed, and the Drive field indicates where the media is located.

Invokes pkgAcquireStatus::MediaChange(Media, Drive) to ask the user to swap disks; informs the subprocess of the result (via 603 Media Changed, with the Failed field set to true if the user cancelled the media change).

◆ MethodConfig()

MethodConfig ( )

Set up the default method parameters.

All fields are initialized to NULL, "", or false as appropriate.

Referenced by Enqueue(), GetConfig(), and ~pkgAcquire().

◆ MethodFailure()

bool MethodFailure ( )

protected

Invoked when the worked process dies unexpectedly.

Waits for the subprocess to terminate and generates an error if it terminated abnormally, then closes and blanks out all file descriptors. Discards all pending messages from the subprocess.

Returns: false.

◆ OutFdReady()

bool OutFdReady ( )

protected

Send any pending commands to the subprocess.

This method will fail if there is no pending output.

Returns: true if all commands were succeeded, false if an error occurred (in which case MethodFailure() will be invoked).

◆ PartialPresent()

APT_PURE unsigned long long PartialPresent ( )

Returns: the amount of data to be fetched that is already present on the filesystem.

References ItemsBegin(), and ItemsEnd().

◆ Queue()

Queue	(	std::string const &	Name,
		pkgAcquire *const	Owner
	)

Create a new Queue.

Parameters

Name	The name of the new queue.
Owner	The download process that owns the new queue.

◆ QueueItem()

bool QueueItem ( pkgAcquire::Queue::QItem * Item )

Tell the subprocess to download the given item.

Parameters

Item	the item to queue up.

Returns: true if the item was successfully enqueued.

Queues up a 600 URI Acquire message for the given item to be sent at the next possible moment. Does not flush the output queue.

◆ QueueName()

string QueueName	(	std::string	URI,
		MethodConfig const *&	Config
	)

protected

Determine the fetch method and queue of a URI.

Parameters

	URI	The URI to fetch.
[out]	Config	A location in which to place the method via which the URI is to be fetched.

Returns: the string-name of the queue in which a fetch request for the given URI should be placed.

References Config, HashStringList::FileSize(), GetConfig(), QueueAccess, and Queues.

Referenced by Enqueue().

◆ QueueURI()

virtual bool QueueURI ( ItemDesc & Item )

protectedvirtual

Insert this item into its owner's queue.

The method is designed to check if the request would end in an IMSHit and if it determines that it would, it isn't queueing the Item and instead sets it to completion instantly.

Parameters

Item	Metadata about this item (its URI and description).

Returns: true if the item was inserted, false if IMSHit was detected

◆ ReadMessages()

bool ReadMessages ( )

protected

Retrieve any available messages from the subprocess.

The messages are retrieved as in ReadMessages(), and MethodFailure() is invoked if an error occurs; in particular, if the pipe to the subprocess dies unexpectedly while a message is being read.

Returns: true if the messages were successfully read, false otherwise.

◆ Rename()

bool Rename	(	std::string const &	From,
		std::string const &	To
	)

protected

Rename a file without modifying its timestamp.

Many item methods call this as their final action.

Parameters

From	The file to be renamed.
To	The new name of From. If To exists it will be overwritten. If From and To are equal nothing happens.

◆ RenameOnError()

bool RenameOnError ( RenameOnErrorState const state )

protected

Rename failed file and set error.

Parameters

state respresenting the error we encountered

◆ Run()

pkgAcquire::RunResult Run ( int PulseInterval = 500000 )

Download all the items that have been Add()ed to this download process.

This method will block until the download completes, invoking methods on Log to report on the progress of the download.

Parameters

PulseInterval The method pkgAcquireStatus::Pulse will be invoked on Log at intervals of PulseInterval microseconds.

Returns: the result of the download.

References Cancelled, Continue, Failed, Items, Log, pkgAcquireStatus::Pulse(), Queues, RunFds(), Running, SetFds(), pkgAcquireStatus::Start(), pkgAcquireStatus::Stop(), ToFetch, pkgAcquireStatus::Update, Worker(), and Workers.

◆ RunFds()

bool RunFds	(	fd_set *	RSet,
		fd_set *	WSet
	)

protectedvirtual

Handle input from and output to file descriptors which select() has determined are ready. The default implementation dispatches to all active downloads.

Parameters

RSet	The set of file descriptors that are ready for input.
WSet	The set of file descriptors that are ready for output.

Returns: false if there is an error condition on one of the fds

References Worker(), and Workers.

Referenced by Run().

◆ RunMessages()

bool RunMessages ( )

protected

Parse and dispatch pending messages.

This dispatches the message in a manner appropriate for its type.

Todo:: Several message types lack separate handlers.

See also: Capabilities(), SendConfiguration(), MediaChange()

◆ SendConfiguration()

bool SendConfiguration ( )

protected

Send a 601 Configuration message (containing the APT configuration) to the subprocess.

The APT configuration will be send to the subprocess in a message of the following form:

601 Configuration
Config-Item: Fully-Qualified-Item=Val
Config-Item: Fully-Qualified-Item=Val
...

Returns: true if the command was successfully sent, false otherwise.

◆ SetActiveSubprocess()

void SetActiveSubprocess ( std::string const & subprocess )

Set the name of the current active subprocess.

◆ SetFds()

void SetFds	(	int &	Fd,
		fd_set *	RSet,
		fd_set *	WSet
	)

protectedvirtual

Build up the set of file descriptors upon which select() should block.

The default implementation inserts the file descriptors corresponding to active downloads.

Parameters

[out]	Fd	The largest file descriptor in the generated sets.
[out]	RSet	The set of file descriptors that should be watched for input.
[out]	WSet	The set of file descriptors that should be watched for output.

References InFd, OutFd, Worker(), and Workers.

Referenced by Run().

◆ ShortDesc()

virtual std::string ShortDesc ( ) const

virtual

Short item description.

Returns: a brief description of the object being fetched.

◆ Shutdown()

bool Shutdown ( bool Final )

Shut down the worker process associated with this queue.

Parameters

Final If true, then the process is stopped unconditionally. Otherwise, it is only stopped if it does not need cleanup as indicated by the pkgAcqMethod::NeedsCleanup member of its configuration.

Returns: true.

◆ Start() [1/2]

bool Start ( )

Start up the worker and fill in Config.

Reads the first message from the worker, which is assumed to be a 100 Capabilities message.

Returns: true if all operations completed successfully.

◆ Start() [2/2]

virtual void Start	(	std::string const &	Message,
		unsigned long long const	Size
	)

virtual

Invoked when the worker starts to fetch this object.

Parameters

Message	RFC822-formatted data from the worker process. Use LookupTag() to parse it.
Hashes	The expected hashes of the object being fetched.

See also: pkgAcqMethod

◆ Startup()

bool Startup ( )

Start the worker process associated with this queue.

If a worker process is already associated with this queue, this is equivalent to calling Cycle().

Returns: true if the startup was successful.

◆ TotalNeeded()

APT_PURE unsigned long long TotalNeeded ( )

Returns: the total size in bytes of all the items included in this download.

References ItemsBegin(), and ItemsEnd().

◆ UriBegin()

pkgAcquire::UriIterator UriBegin ( )

Get the head of the list of enqueued item URIs.

This iterator will step over every element of every active queue.

References Queues, and UriIterator().

◆ UriIterator()

UriIterator ( pkgAcquire::Queue * Q )

explicit

Create a new UriIterator.

Parameters

Q	The queue over which this UriIterator should iterate.

Referenced by UriBegin(), and UriEnd().

◆ VerifyDone()

virtual bool VerifyDone	(	std::string const &	Message,
		pkgAcquire::MethodConfig const *const	Cnf
	)

virtual

Invoked by the acquire worker to check if the successfully fetched object is also the objected we wanted to have.

Note that the object might not have been written to DestFile; check for the presence of an Alt-Filename entry in Message to find the file to which it was really written.

This is called before Done is called and can prevent it by returning false which will result in Failed being called instead.

You should prefer to use this method over calling Failed() from Done() as this has e.g. the wrong progress reporting.

Parameters

Message	Data from the acquire method. Use LookupTag() to parse it.
Cnf	The method via which the object was fetched.

See also: pkgAcqMethod

◆ Worker() [1/2]

Worker ( MethodConfig * Config )

explicit

Create a new Worker that should just retrieve information about the fetch method.

Nothing in particular forces you to refrain from actually downloading stuff, but the various status callbacks won't be invoked.

Parameters

Config A location in which to store information about the fetch method.

◆ Worker() [2/2]

Worker	(	Queue *	OwnerQ,
		MethodConfig *	Config,
		pkgAcquireStatus *	Log
	)

Create a new Worker to download files.

Parameters

OwnerQ	The queue into which this worker should be placed.
Config	A location in which to store information about the fetch method.
Log	The download progress indicator that should be used to report the progress of this worker.

Referenced by GetConfig(), Remove(), Run(), RunFds(), SetFds(), and WorkerStep().

◆ WorkersBegin()

Worker * WorkersBegin ( )

inline

Get the first Worker object.

Returns: the first active worker in this download process.

Referenced by pkgAcquireStatus::Pulse().

◆ WorkerStep()

pkgAcquire::Worker * WorkerStep ( Worker * I )

Advance to the next Worker object.

Returns: the worker immediately following I, or NULL if none exists.

References Worker().

Referenced by pkgAcquireStatus::Pulse().

Member Data Documentation

◆ Access

std::string Access

protected

The access method to be used by this worker.

The name of this acquire method (e.g., http).

Todo:: Doesn't this duplicate Config->Access?

Referenced by GetConfig().

◆ Complete

bool Complete

If true, the entire object has been successfully fetched.

Subclasses should set this to true when appropriate.

◆ Configs

MethodConfig* Configs

protected

The head of the list of acquire method configurations.

Each protocol (http, ftp, gzip, etc) via which files can be fetched can have a representation in this list. The configuration data is filled in by parsing the 100 Capabilities string output by a method on startup (see pkgAcqMethod::pkgAcqMethod and pkgAcquire::GetConfig).

Todo:: why a hand-managed config dictionary instead of std::map?

Referenced by GetConfig(), and ~pkgAcquire().

◆ Debug

bool Debug

protected

If true, debugging output will be sent to std::clog.

Referenced by Dequeue(), and Enqueue().

◆ ExpectedAdditionalItems

unsigned int ExpectedAdditionalItems

The number of additional fetch items that are expected once this item is done.

Some items like pkgAcqMeta{Index,Sig} will queue additional items. This variable can be set by the methods if it knows in advance how many items to expect to get a more accurate progress.

◆ ID

unsigned long ID

A client-supplied unique identifier.

This field is initialized to 0; it is meant to be filled in by clients that wish to use it to uniquely identify items.

APT progress reporting will store an ID there as shown in "Get:42 …"

◆ InFd

int InFd

protected

A file descriptor connected to the standard output of the subprocess.

Used to read messages and data from the subprocess.

Referenced by SetFds().

◆ InReady

bool InReady

protected

Set to true if the worker is in a state in which it might generate data or command responses.

Todo:: Is this right? It's a guess.

◆ Items [1/2]

std::vector<Item *> Items

protected

A list of items to download.

This is built monotonically as items are created and only emptied when the download shuts down.

Referenced by Add(), Clean(), Remove(), Run(), and Shutdown().

◆ Items [2/2]

QItem* Items

protected

The head of the list of items contained in this queue.

Todo:: why a by-hand list instead of an STL structure?

◆ Local

bool Local

If true, the URI of this object is "local".

The only effect of this field is to exclude the object from the download progress indicator's overall statistics.

◆ Log

pkgAcquireStatus * Log

protected

The download progress indicator to which progress messages should be sent.

The progress indicator for this download.

Referenced by Enqueue(), Run(), and pkgAcqMetaBase::VerifyVendor().

◆ NeedsCleanup

bool NeedsCleanup

If true, the subprocess has to carry out some cleanup actions before shutting down.

For instance, the cdrom method needs to unmount the CD after it finishes.

◆ Next

MethodConfig* Next

The next link on the acquire method list.

Todo:: Why not an STL container?

◆ NextQueue

Worker* NextQueue

protected

The next link on the Queue list.

Todo:: This is always NULL; is it just for future use?

◆ OutFd

int OutFd

protected

A file descriptor connected to the standard input of the subprocess.

Used to send commands and configuration data to the subprocess.

Referenced by SetFds().

◆ OutQueue

std::string OutQueue

protected

Buffers pending writes to the subprocess.

Todo:: Wouldn't a std::dequeue be more appropriate?

◆ OutReady

bool OutReady

protected

Set to true if the worker is in a state in which it is legal to send commands to it.

Todo:: Is this right?

◆ QueueCounter

unsigned int QueueCounter

The number of fetch queues into which this item has been inserted.

There is one queue for each source from which an item could be downloaded.

See also: pkgAcquire

◆ Queues

Queue* Queues

protected

The head of the list of active queues.

Todo:: why a hand-managed list of queues instead of std::list or std::set?

Referenced by Bump(), Dequeue(), Enqueue(), QueueName(), Run(), Shutdown(), and UriBegin().

◆ Workers [1/2]

Worker* Workers

protected

The head of the list of active workers.

Todo:: why a hand-managed list of workers instead of std::list or std::set?

Referenced by Add(), Remove(), Run(), RunFds(), and SetFds().

◆ Workers [2/2]

pkgAcquire::Worker* Workers

protected

The head of the list of workers associated with this queue.

Todo:: This is plural because support exists in Queue for multiple workers. However, it does not appear that there is any way to actually associate more than one worker with a queue.

Todo:: Why not just use a std::set?

The documentation for this struct was generated from the following files:

apt-pkg/acquire-item.h
apt-pkg/acquire.h
apt-pkg/acquire-worker.h
apt-pkg/acquire.cc

Classes

Public Types

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Types

Protected Member Functions

Protected Attributes

Friends

Detailed Description

Member Enumeration Documentation

◆ ItemState

◆ QueueStrategy

◆ RunResult

Constructor & Destructor Documentation

◆ ~Worker()

◆ ~pkgAcquire()

◆ ~Queue()

Member Function Documentation

◆ Bump() [1/2]

◆ Bump() [2/2]

◆ Capabilities()

◆ Clean()

◆ CleanLists()

◆ Construct()

◆ Custom600Headers()

◆ Cycle()

◆ Dequeue()

◆ DescURI()

◆ Done()

◆ Enqueue() [1/2]

◆ Enqueue() [2/2]

◆ Failed()

◆ FetchNeeded()

◆ FindItem()

◆ GetConf()

◆ GetConfig()

◆ GetExpectedHashes()

◆ GetLock()

◆ GetOwner()

◆ HashesRequired()

◆ HashSum()

◆ InFdReady()

◆ IsTrusted()

◆ Item()

◆ ItemDone() [1/2]

◆ ItemDone() [2/2]

◆ ItemStart()

◆ MediaChange()

◆ MethodConfig()

◆ MethodFailure()

◆ OutFdReady()

◆ PartialPresent()

◆ Queue()

◆ QueueItem()

◆ QueueName()

◆ QueueURI()

◆ ReadMessages()

◆ Rename()

◆ RenameOnError()

◆ Run()

◆ RunFds()

◆ RunMessages()

◆ SendConfiguration()

◆ SetActiveSubprocess()

◆ SetFds()

◆ ShortDesc()

◆ Shutdown()

◆ Start() [1/2]

◆ Start() [2/2]

◆ Startup()

◆ TotalNeeded()

◆ UriBegin()

◆ UriIterator()

◆ VerifyDone()

◆ Worker() [1/2]

◆ Worker() [2/2]

◆ WorkersBegin()

◆ WorkerStep()

Member Data Documentation