After upgrade, what do you do about "removed" and "obsolete" packages ?

[Thomas Schmitt]

Hi,

today i upgraded a Debian 11 system to 12 and am now scratching my head
over the final steps as described in
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#purge-removed-packages
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#obsolete

The command
apt list '~c'
shows 9 "removed packages":
fuse/stable 2.9.9-6+b1 amd64 [residual-config]
libreoffice-avmedia-backend-gstreamer/now 1:7.0.4-4+deb11u9 amd64 [residual-config]
linux-image-4.19.0-17-amd64/now 4.19.194-3 amd64 [residual-config]
linux-image-4.19.0-20-amd64/now 4.19.235-1 amd64 [residual-config]
linux-image-4.19.0-9-amd64/now 4.19.118-2+deb10u1 amd64 [residual-config]
python-twisted-core/now 18.9.0-3 all [residual-config]
python/now 2.7.16-1 amd64 [residual-config]
wicd-daemon/now 1.7.4+tb2-6 all [residual-config]
wicd-gtk/now 1.7.4+tb2-6 all [residual-config]
What does "[residual-config]" mean ?
The man page of apt is quite sparse. Is there something more detailed
available online ?

apt list '~o'
shows 220 "obsolete packages", of which at least one is a commercial
non-Debian package which is on the machine for commercial reasons.
Others are gcc-{8,9,10}, hfsprogs, linux-image-* from Debian 10 and from
my own kernel experiments, and lots of stuff of which i have no clue.

I wonder how others sift through such a list and decide what to do.


Have a nice day :)

Thomas

[Marco Moock]

Post by Thomas Schmitt
today i upgraded a Debian 11 system to 12 and am now scratching my
head over the final steps as described in
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#purge-removed-packages
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#obsolete

Packages have dependencies. Those will be marked as automatically
installed. They can be removed if no other package depends on them.

You can do that with the autopurge/autoremove apt command.

Be aware: If you install software beyond apt/dpkg that depends on files
in installed packages, you need to mark them as manually installed to
avoid being removed by autoremove. dpkg doesn't care about stuff
manually installed.

Post by Thomas Schmitt
What does "[residual-config]" mean ?

Packages include system-wide configuration files. If packages are
removed, this configuration will not be deleted. You need to purge such
packages to remove it.

[The Wanderer]

Post by Marco Moock

Post by Thomas Schmitt
What does "[residual-config]" mean ?

Packages include system-wide configuration files. If packages are
removed, this configuration will not be deleted. You need to purge
such packages to remove it.

On brief analysis, it looks like this reflects the same status which is
reported in the Status line of the output of 'dpkg -s PACKAGENAME' with
the string "deinstall ok config-files". YMMV, but I find that intuitive
enough: the package used to be installed, and isn't anymore, but its
config files have been left behind.

(Note that if you have the package installed from multiple
architectures, you will apparently need to specify the arch qualifier
suffix to the package name in order for the command to not error out.)

I don't remember using that dpkg command very often, but I do remember
seeing that string often enough in the past, so there are probably other
commands which will also report it if applicable.

[Thomas Schmitt]

Hi,

i wonder why none of the electricians on this list has an anecdote to
share about dealing with "obsolete" packages after upgrade.
No triumphs, defeats, or global catastrophes ?

Post by Marco Moock

Post by Thomas Schmitt
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.h

tml#purge-removed-packages

Post by Marco Moock

Post by Thomas Schmitt
What does "[residual-config]" mean ?

Packages include system-wide configuration files. If packages are
removed, this configuration will not be deleted. You need to purge such
packages to remove it.

So the smaller list of packages can be dealt with what the upgrade
instructions propose:
apt purge '~c'


There remains the list of 220 "obsolete" packages.

Post by Marco Moock

Post by Thomas Schmitt
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#obsolete

Packages have dependencies. Those will be marked as automatically
installed. They can be removed if no other package depends on them.

But several of those packages were surely installed manually by me via
dpkg -i after being made made as descibed in
https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-building
The predicate "obsolete" is not the same as "automatically installed".
I understand that obsolete means having no successor package in the
upgraded Debian release.

How could i get a list of only the automatically installed obsolete
packages ?
(I still did not find any documentation about the '~c' or '~o' with
"apt list".)

Post by Marco Moock
Be aware: If you install software beyond apt/dpkg that depends on files
in installed packages, you need to mark them as manually installed to
avoid being removed by autoremove.

Google leads me to apt-mark for that purpose.
But i am not sure whether the commercial package which i have to keep
will be preserved with "apt autoremove".
Is there a way to do a dry run which only tells what would happen if i
were more courageous ?


Have a nice day :)

Thomas

[Mike Kupfer]

Post by Thomas Schmitt
But i am not sure whether the commercial package which i have to keep
will be preserved with "apt autoremove".
Is there a way to do a dry run which only tells what would happen if i
were more courageous ?

When I use "apt autoremove", I am given a list of proposed removals and
a prompt about whether I want to proceed.

cheers,
mike

[Max Nikulin]

Post by Thomas Schmitt
How could i get a list of only the automatically installed obsolete
packages ?
(I still did not find any documentation about the '~c' or '~o' with
"apt list".)

apt-patterns(7) and dpkg(1). Apt can not distinguish packages installed
by dpkg directly and by apt but not present in any configured repository
any more. Both are "obsolete".

[Thomas Schmitt]

Hi,

Post by Mike Kupfer

Post by Thomas Schmitt
But i am not sure whether the commercial package which i have to keep
will be preserved with "apt autoremove".
Is there a way to do a dry run which only tells what would happen if i
were more courageous ?

When I use "apt autoremove", I am given a list of proposed removals and
a prompt about whether I want to proceed.

Good to know that there are safeguards when i finally remove some of the
"obsolete" packages.

Post by Mike Kupfer

Post by Thomas Schmitt
How could i get a list of only the automatically installed obsolete
packages ?
(I still did not find any documentation about the '~c' or '~o' with
"apt list".)

apt-patterns(7)

Wow. What kind of programming language can have inspired the developers
to define such a syntax ?
But hey, at least there is logic provided. \o/

So i try

apt list '?installed ?obsolete ?automatic'

This narrows the list from 220 to 192 packages.
Even better, i don't have to diff the lists but can see the 28 other
obsolete packages by

apt list '?installed ?obsolete !?automatic'

Among them are "hfsprogs", the self-made kernels, and the commercial
package which i need to keep.


Next documenation riddle is what the word "local" means in output lines
like

linux-image-5.10.0-rc2-ts/now 5.10.0-rc2-ts-37 amd64 [installed,local]

(I may have missed something in the man pages of dpkg and dpkg-query, but
their occurences of the word "local" do not look like related to the info
from "apt list".)


Have a nice day :)

Thomas

[Florent Rougon]

Hi Thomas,

Post by Thomas Schmitt
Next documenation riddle is what the word "local" means in output lines
like
linux-image-5.10.0-rc2-ts/now 5.10.0-rc2-ts-37 amd64 [installed,local]

I don't use this but guess it is as in aptitude, where “obsolete/local
packages” are packages that are installed (from dpkg's POV) but not
available from any of the repositories scanned in the last 'apt update'
run.

This happens in particular with:

- packages that used to be in a repo seen by 'apt update' (often, you
installed said packages at that time), but are not included in your
current apt sources (/etc/apt/sources.list,
/etc/apt/sources.list.d/); this usually happens between Debian
releases for some packages shipped by Debian;

- packages that are not in any of the repos seen by 'apt update' and
that you installed from .deb with 'dpkg -i' (I believe the apt
command line tool can also do this); for instance, local packages
you prepared yourself but didn't bother to put in an apt repository.

Regards

[Thomas Schmitt]

Hi,

Post by Florent Rougon

Post by Thomas Schmitt
Next documenation riddle is what the word "local" means in output lines
like
linux-image-5.10.0-rc2-ts/now 5.10.0-rc2-ts-37 amd64 [installed,local]

I don't use this but guess it is as in aptitude, where “obsolete/local
packages” are packages that are installed (from dpkg's POV) but not
available from any of the repositories scanned in the last 'apt update'
run.

So "local" would be just another word for "obsolete" ?

Indeed, the output of

apt list '?installed !?obsolete' | grep local

shows only some packages with "locale" in their name, but none with
"local" in the []-brackets. On the other hand

apt list '?installed ?obsolete' | grep -v 'local]$'

shows no packages, i.e. all lines of obsolete packages end by "local]".


Nevertheless it would be nice to find documentation about this kind of
info in the output of "apt list".


Have a nice day :)

Thomas

[Florent Rougon]

Post by Thomas Schmitt
So "local" would be just another word for "obsolete" ?

My understanding is that “obsolete” and “local” may mean different
things to the person who installed the packages (“obsolete” would
correspond to the first item of the list at the end of my previous post,
“local” to the second one); however, apt and aptitude can't distinguish
between them: both categories are comprised of “packages that are
installed but not available from the sources scanned during the last
'apt update' run (or 'aptitude update').

I believe someone already wrote something along these lines in this
thread (maybe Max). In aptitude, the packages in question are all
grouped in the category named “Obsolete and Locally Created Packages”,
IMHO because there is no good way to programmatically distinguish
between them.

(A private package could very well be made available and installed from
a private repository; or alternatively, installed with 'dpkg -i' without
ever being put in an apt repository; therefore “has been installed in
the past from an apt repository” is not a good criterion to distinguish
between “obsolete packages” and “local” ones.)

Note: I mentioned private packages to simplify wording, but the
“Obsolete and Locally Created Packages” category would also
contain packages that users sometimes download from third-party
websites[1], installing them with 'dpkg -i' or the 'apt' command
line tool, without adding any repository to their
sources.list(.d). All these are “local packages” from my POV.

Regards

[1] Printer drivers in .deb form, libdvdcss stuff, etc. (make sure the
source is trustworthy!)

[Curt]

Post by Thomas Schmitt
Nevertheless it would be nice to find documentation about this kind of
info in the output of "apt list".

I found this from an old post about Synaptic (the apt front-end), in the latter's
"help page":

Obsolete or locally installed - Display only packages that are not (for longer)
(sic) included in one of the specified repositories.

I guess the terminology is intended to cover all conceivable cases of why a
package isn't found in the repositories, in the event the user grew confused by
one or the other term that wasn't relevant to her situation, although for APT
the two cases are indistinguishable.

Post by Thomas Schmitt
Have a nice day :)
Thomas

--

[Thomas Schmitt]

Hi,

just for the archive:

I think i found the source code which emits the "[...]" strings of
apt-list:

https://sources.debian.org/src/apt/2.9.4/apt-private/private-output.cc/#L292

The possible status strings are:

[installed,upgradable to: ...]
[installed,local]
[installed,auto-removable]
[installed,automatic]
[installed]
[upgradable from: ...]"
[residual-config]

More seem not to exist.

Their meaning has to be guessed from the names of methods, variables, and
constants in the code. (I believe "pkgCache::Flag::Auto" is a constant and
not an overloaded monster car. With C++ one never knows what's behind a
name.)

if (P->CurrentVer != 0)
{
if (P.CurrentVer() == V)
{
if (state.Upgradable() && state.CandidateVer != NULL)
strprintf(StatusStr, _("[installed,upgradable to: %s]"),
else if (V.Downloadable() == false)
StatusStr = _("[installed,local]");
else if(V.Automatic() == true && state.Garbage == true)
StatusStr = _("[installed,auto-removable]");
else if ((state.Flags & pkgCache::Flag::Auto) == pkgCache::Flag::Auto)
StatusStr = _("[installed,automatic]");
else
StatusStr = _("[installed]");
}
else if (state.CandidateVer == V && state.Upgradable())
strprintf(StatusStr, _("[upgradable from: %s]"),
InstalledVerStr.c_str());
}
else if (V.ParentPkg()->CurrentState == pkgCache::State::ConfigFiles)
StatusStr = _("[residual-config]");

---------------------------------------------------------------------
About my post-upgrade activities:

I ran the command that is proposed in
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.html#purge-removed-packages
apt purge '~c'

It flooded me with package names under the headline

The following packages were automatically installed and are no longer required:
...
Use 'apt autoremove' to remove them.

Then it offered me a list with slightly frightening wildcards:

The following packages will be REMOVED:
fuse* libreoffice-avmedia-backend-gstreamer* linux-image-4.19.0-17-amd64*
linux-image-4.19.0-20-amd64* linux-image-4.19.0-9-amd64* python*
python-twisted-core* wicd-daemon* wicd-gtk*

After my confirmation it purged the 9 configurations which were reported
by "apt list '~c'"
Purging configuration files for fuse (2.9.9-5) ...
...
Purging configuration files for python (2.7.16-1) ...
and as last line said
Processing triggers for dbus (1.14.10-1~deb12u1) ...

Somewhat mistrusting about the removal of "python" i asked apt-file from
where my current /usr/bin/python stems. Answer: python-is-python3 .
(As side result i now wonder how the result of /usr/bin/python3-pasteurize
might taste and how long it stays fresh.)


I will probably run "apt autoremove" after verifying that the few
worthy local packages are not in the list proposed for autoremoval.


Have a nice day :)

Thomas

[Florent Rougon]

Post by Thomas Schmitt
fuse* libreoffice-avmedia-backend-gstreamer* linux-image-4.19.0-17-amd64*
linux-image-4.19.0-20-amd64* linux-image-4.19.0-9-amd64* python*
python-twisted-core* wicd-daemon* wicd-gtk*

AFAIK, these are not wildcards; each star appended to a package name
indicates that the package is going to be purged (no config files left
afterwards), as opposed to simply removed (leaving configuration files
in place).

Post by Thomas Schmitt
I will probably run "apt autoremove" after verifying that the few
worthy local packages are not in the list proposed for autoremoval.

tl;dr: aptitude praise
~~~~~~~~~~~~~~~~~~~~~~

FWIW, removal of “obsolete or local” packages is easily done
interactively in aptitude: you go the the corresponding section of the
main screen, hit Enter, etc. The [ key recursively unfolds a section
(use ] to fold it back). You ask to purge a package by typing _
(removing with -, as in the venerable dselect).

For actually obsolete packages, doing so will occasionally trigger a
“dependency problem” because another package that depends on it hasn't
been marked as “to be removed or purged” yet. But normally, that other
package is also obsolete, so it *will* be marked shortly after when you
get to it. So basically, once you've gone through all the obsolete
packages marking each one as “to be purged”, having only left intact the
local ones you do want to keep, there should be no dependency problem to
resolve. ⇒ Hit g (for “go”), check the preview, hit g again if it looks
fine, otherwise q.

Note that the preview (of what is going to be done) is shown in a new
tab (yes, these are tabs, you can switch between them with F6), and that
tab gets closed if you cancel the operation with q. Also, you can act
directly in the Preview tab to unmark an operation, etc. And you can
undo with Ctrl-u, including outside the Preview tab.

Generally and especially for this kind of use (removing obsolete
packages that are still installed), I find that the following lines are
a must-have in /etc/apt/apt.conf:

// Similar to dselect
Aptitude::UI::Advance-On-Action "true";

(I also like “Aptitude::Auto-Upgrade "true";” but it is irrelevant
here.)

apt and aptitude have different algorithms for handling upgrades, so:
for stable-to-next-stable upgrades, I do stick to what the Release Notes
recommend. In most other situations where there isn't a huge number of
packages to upgrade, I find that aptitude does a great job:
- interactive control over what is going to be done;
- visualization of packages marked as auto-installed vs. those not
marked as such (and you can flip this bit using m or M);
- interactive, regexp-based search (with powerful features if you look
up the syntax in the manual);
- interactive package list limited by a user-defined filter (this is
Limit Display in the Search menu);
- interactive inspection of the (deps a package + other control
fields) with Enter; of its reverse deps with the r key (and you can
quicky recurse in order to find why you need to have this pkg
installed);
- works in a terminal;
- etc.

There is the occasional crash, fortunately I've never seen one happen
while dpkg was installing, removing, etc., so the crashes I've seen were
all rather harmless (restart aptitude and proceed again). The worst I
had was on sid during the time_t transition: at some point, aptitude
couldn't start without crashing; however, after upgrading a few packages
with 'apt', it all went back into order. :-)

Regards

[Florent Rougon]

Post by Florent Rougon
FWIW, removal of “obsolete or local” packages is easily done
interactively in aptitude: you go the the corresponding section of the
main screen, hit Enter, etc. The [ key recursively unfolds a section
(use ] to fold it back). You ask to purge a package by typing _
(removing with -, as in the venerable dselect).

Forgot to say: one can perform the operation (remove with -, purge with
_) on a whole section at once by pressing the key on the section title,
where by “section” I mean: a foldable group of packages (e.g.: admin,
kernel, libsdevel, libs, etc., plus actual Debian sections: main,
contrib, nonfree, and presumably also the new non-free-firmware).

So, for instance, a whole bunch of obsolete library pakages can be
marked all at once for purge with a single _ keypress (this doesn't
exempt one from _reading_ the list of packages marked with this
keypress—make sure you didn't overlook something!).

Regards

[Thomas Schmitt]

Hi,

Post by Florent Rougon
AFAIK, these are not wildcards; each star appended to a package name
indicates that the package is going to be purged

At least it is a good way to catch the attention of the apt operator. :))

Post by Florent Rougon
tl;dr: aptitude praise

Thanks for this description of a real world procedure.
Now i know at least that i am not the only one who cares about the
post-upgrade steps in the manual. I already began to think that everybody
lets the surplus packages rot in the dark.


Have a nice day :)

Thomas

[songbird]

Thomas Schmitt wrote:
...

Post by Thomas Schmitt
Thanks for this description of a real world procedure.
Now i know at least that i am not the only one who cares about the
post-upgrade steps in the manual. I already began to think that everybody
lets the surplus packages rot in the dark.

i have been running the same general routine for
years. every morning i boot up, run the updates
for apt (but i use apt-get out of habit) and then
do the updates that make sense. since i am running
testing i may delay some updates for a period of
time until issues look to be resolved.

if there are packages that are no longer needed
i will usually leave them in place until after i've
made sure i don't need them any more (here or there
i have had to downgrade to fix an issue so i don't
want to have to recreate a configuration file) before
i purge them from my system, but it may only be a
few days where i don't have the purged. i also used
to run orphaner, but i've not used that in quite a
long time and it doesn't even appear in the archive
any more.

after upgrading i reboot. i want to know something
is broken when i have time to fix it instead of
finding out when i don't have time. i do keep a
bootable stable partition though for emergencies and
comparison.

once in a while i remove any unneeded downloaded
packages from my apt cache so i'm not wasting too
much space on my system, but that's once a month or
so when things seem to be quiet enough that i'm not
at risk of removing a package i might need to
downgrade.

Post by Thomas Schmitt
Have a nice day :)

:)

Post by Thomas Schmitt
Thomas

songbird

[Max Nikulin]

Post by Thomas Schmitt

Post by Max Nikulin
apt-patterns(7)

Wow. What kind of programming language can have inspired the developers
to define such a syntax ?

https://blog.jak-linux.org/2019/08/15/apt-patterns/

"apt list" has some limitations in comparison to "aptitude search". I
have no idea what was the source of inspiration for aptitude, but this
query language appeared perhaps a quarter of century ago.

https://www.debian.org/doc/manuals/aptitude/ch02s04s05.en.html

For me it is hard to compose a query beyond trivial ones. My adventures
with "apt list":
Re: List packages from non-default repositories. Wed, 4 Oct 2023
17:26:47 +0700
https://lists.debian.org/msgid-search/ufjel7$l9m$***@ciao.gmane.io

However I did not post last variant of the query. It is for aptitude.

As to obsolete vs. local packages, my guess is that apt may label some
version as obsolete if another version of the same package is still
available from some repository. Otherwise it is local.

[Thomas Schmitt]

Hi,

Post by Max Nikulin

Post by Thomas Schmitt
What kind of programming language can have inspired the developers
to define such a syntax ?

https://blog.jak-linux.org/2019/08/15/apt-patterns/

This points to aptitude. The package description of aptitude says
"mutt-like syntax for matching packages". Indeed
https://www.sendmail.org/~ca/email/mutt/manual-4.html
has in its second half some lines which resemble apt-patterns.

Post by Max Nikulin
As to obsolete vs. local packages, my guess is that apt may label some
version as obsolete if another version of the same package is still
available from some repository. Otherwise it is local.

To me it seems that apt-patterns simply calls "obsolete" what apt-list
then marks in its output as "[... local]".
Obviously these terms refer to different reasons why a package is not
found in the official repos. But these reasons seem to be
indistinguishable. So in the end both terms depict the same status.


Have a nice day :)

Thomas

[David Wright]

Post by Thomas Schmitt
i wonder why none of the electricians on this list has an anecdote to
share about dealing with "obsolete" packages after upgrade.
No triumphs, defeats, or global catastrophes ?

Nowadays I install new releases from scratch, helped by the fact that
for years I've always had two systems on each machine, the current and
the previous. (/home is shared.) That tends to limit cruft as well.

Post by Thomas Schmitt

Post by Thomas Schmitt
https://www.debian.org/releases/stable/amd64/release-notes/ch-upgrading.en.h

tml#purge-removed-packages

Post by Thomas Schmitt
What does "[residual-config]" mean ?

It is odd that the terminology used in the output differs from that
used to provoke it, as in:

$ apt list '?config-files'
Listing... Done
mlocate/oldstable,now 0.26-5 amd64 [residual-config]
$

Both man apt-patterns and aptitude's Search Term Reference ought to
include the bracketed items if there's no intention to unify terms.
Perhaps it's related to the tendency to underdocument the output from
programs.

Post by Thomas Schmitt
The predicate "obsolete" is not the same as "automatically installed".
I understand that obsolete means having no successor package in the
upgraded Debian release.

"Obsolete" is an unfortunately loaded word. I think aptitude expresses
it a bit more clearly: "This term matches any installed package which
is not available in any version from any archive. These packages
appear as “Obsolete or Locally Installed” in the visual interface."

$ apt list '?installed ?obsolete'
Listing... Done
xtoolwait/now 1.3-6.2 amd64 [installed,local]
yt-dlp/now 2024.05.26-1 all [installed,local]
$

The first is from squeeze, the second from trixie (hardly "obsolete"),
both installed with apt-get fullpath (previously I'd have used dpkg -i).

Post by Thomas Schmitt
Is there a way to do a dry run which only tells what would happen if i
were more courageous ?

Both apt* and dpkg have --no-act --dry-run --simulate to prevent
acting. (apt* has additional synonyms -s --just-print --recon.)
With dpkg, it's safest to place the option first, as it only
protects what follows it in the command line.

Typically you can also not be root to help protect yourself,
as in:

$ apt-get -s remove libc6
NOTE: This is only a simulation!
apt-get needs root privileges for real execution.
Keep also in mind that locking is deactivated,
so don't depend on the relevance to the real current situation!
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
[ … ]
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
[ … ]
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
[ … ]
0 upgraded, 0 newly installed, 1702 to remove and 0 not upgraded.
[ … ]

Cheers,
David.