where are the actual ".deb" packages? all I see are "pkgcache.bin" and "srcpkgcache.bin" . . .

[Albretch Mueller]

Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.
apt-rdepends nicely lists all dependencies ordered as a depth first
graph, but I am getting "pkgcache.bin" and "srcpkgcache.bin" files for
all package names.

I found a nice explanation of what seems to be all there is to be
known about apt (any other documentation you would recommend?):

https://www.juliensobczak.com/inspect/2021/05/15/linux-packages-under-the-hood.html

but I don't see an answer to my question. Where are or how do I get
the ".deb" package out of those binary files?

_DEB_PKG="gimp"
_ARCH=$(dpkg --print-architecture)
_DT=$(date +%Y%m%d%H%M%S)
_LOG="apt-rdepends_${_DT}_${_DEB_PKG}_${_ARCH}.log"
echo "// __ \$_LOG: |${_LOG}|"
###
time apt-rdepends "${_DEB_PKG}:${_ARCH}" | grep --extended-regexp
'^[a-zA-Z0-9]' | tac > "${_LOG}" 2>&1

ls -l "${_LOG}"
wc -l "${_LOG}"

_DEB_PKG_2DL=$(head -n 1 "${_LOG}")
echo "// __ \$_DEB_PKG_2DL: |${_DEB_PKG_2DL}|"
_DL_DIR="$(pwd)/${_DT}/${_DEB_PKG_2DL}"
mkdir --parents --verbose "${_DL_DIR}"
apt --option APT::Architecture="${_ARCH}" --option
Dir::Cache="${_DL_DIR}" download "${_DEB_PKG_2DL}"
ls -l "${_DL_DIR}"
file "${_DL_DIR}/"*.*


// __ $_LOG: |apt-rdepends_20230226170006_gimp_amd64.log|
-rw-r--r-- 1 user user 3254 Feb 26 17:00
apt-rdepends_20230226170006_gimp_amd64.log
270 apt-rdepends_20230226170006_gimp_amd64.log
// __ $_DEB_PKG_2DL: |xdg-utils|
mkdir: created directory '/home/user/20230226170006'
mkdir: created directory '/home/user/20230226170006/xdg-utils'
Get:1 http://deb.debian.org/debian bullseye/main amd64 xdg-utils all
1.1.3-4.1 [75.5 kB]
Fetched 75.5 kB in 1s (143 kB/s)
total 67616
-rw-r--r-- 1 user user 34678812 Feb 26 17:00 pkgcache.bin
-rw-r--r-- 1 user user 34555088 Feb 26 17:00 srcpkgcache.bin
/home/user/20230226170006/xdg-utils/pkgcache.bin: APT cache data,
version 16.0, little-endian, 97918 packages, 87142 versions
/home/user/20230226170006/xdg-utils/srcpkgcache.bin: APT cache data,
version 16.0, little-endian, 97917 packages, 87141 versions

[Greg Wooledge]

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.

The tricky part here is that the packages on machine A are not necessarily
the same as on machine B. So, installing package P1 on machine A may
need dependencies P2, P3 and P4. But on machine B, it may need dependencies
P3, P4 and P5 instead.

If you ignore that, then the obvious answer is:

A# apt-get --download-only --reinstall install P1
A# cd /var/cache/apt/archives
A# scp *.deb B:/var/cache/apt/archives/
A# ssh ***@B

B# cd /var/cache/apt/archives
B# apt-get install ./P1.deb

That should utilize the cached dependent packages P2, P3, P4 which are
located in /v/c/a/a on machine B.

If additional dependencies are needed, you'll have to fetch those by
hand.

[Albretch Mueller]

Post by Greg Wooledge

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.

The tricky part here is that the packages on machine A are not necessarily
the same as on machine B.

1) I would take care of keeping dependencies in sync;
2) I would like to use a directory of my choosing to keep deb
installation packages;
3) to place all needed files before and keep the order of installation.

What are these binary files used for (their names say something but I
want the deb files out of them)

lbrtchx

[Tim Woodall]

Post by Greg Wooledge

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.

The tricky part here is that the packages on machine A are not necessarily
the same as on machine B. So, installing package P1 on machine A may
need dependencies P2, P3 and P4. But on machine B, it may need dependencies
P3, P4 and P5 instead.
A# apt-get --download-only --reinstall install P1
A# cd /var/cache/apt/archives
A# scp *.deb B:/var/cache/apt/archives/
B# cd /var/cache/apt/archives
B# apt-get install ./P1.deb
That should utilize the cached dependent packages P2, P3, P4 which are
located in /v/c/a/a on machine B.
If additional dependencies are needed, you'll have to fetch those by
hand.

If you want to download all the dependencies (except essential packages)
then you can point to an empty dir.

I do something like:

rm -f ${APT_WORK}/archives/*.deb
APT_CONFIG=${APT_CONFIG} \
DEBIAN_FRONTEND=noninteractive \
apt-get -q -y install -d \
-o APT::Install-Recommends=false \
$@

where APT_CONFIG looks something like:
cat <<EOF >${APT_CONFIG}
APT::Architecture "${ARCH}";
APT::Install-Recommends "false";

Dir::Etc::PreferencesParts "${APT_WORK}/preferences.d";
Dir::Etc::SourceParts "${APT_WORK}/sources.list.d";
Dir::Etc::TrustedParts "${aptdir}/trusted.d";
Dir::Etc::Parts "${aptdir}/conf.d";

Dir::State::Lists "${APT_WORK}/lists";
Dir::State::Status "${APT_WORK}/status";
Dir::Cache::Archives "${APT_WORK}/archives";
Dir::Cache::srcpkgcache "${APT_WORK}/srcpkgcache";
Dir::Cache::pkgcache "${APT_WORK}/pkgcache";
EOF

[Roberto C. Sánchez]

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.
apt-rdepends nicely lists all dependencies ordered as a depth first
graph, but I am getting "pkgcache.bin" and "srcpkgcache.bin" files for
all package names.

You probably want the package apt-offline. From the package
description:

apt-offline can be used to generate a signature on a machine (with no network).

This signature contains all download information required for the APT database
system. This signature file can be used on another machine connected to the
internet (which need not be a Debian box and can even be running windows) to
download the updates.

The downloaded data will contain all updates in a format understood by APT and
this data can be used by apt-offline to update the non-networked machine.

That sounds to me like your exact use case.

Regards,

-Roberto

[davidson]

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.

Like Roberto Sanchez wrote, this sounds like a use-case for the
apt-offline package (which I have not used).

There is also an apt-doc package available that includes a (relatively
old-looking) tutorial called...

Using APT offline
Abstract: This document describes how to use APT in a non-networked
environment, specifically a 'sneaker-net' approach for performing
upgrades.

... which I assume predates the apt-offline package. It looks brief
and to-the-point, and might satisfy some of the curiosity you have
about how APT works.

Post by Albretch Mueller
apt-rdepends nicely lists all dependencies ordered as a depth first
graph,

Sure, sort of like

$ apt-cache --recurse rdepends xterm # or whatever

Post by Albretch Mueller
but I am getting "pkgcache.bin" and "srcpkgcache.bin" files for all
package names.

I suspect that what is happening, which makes you say this, is that
you are running a script that is looking in the wrong directory for
deb files.

If I am correct, then the answer is to fix your script. See below.

Post by Albretch Mueller
I found a nice explanation of what seems to be all there is to be
https://www.juliensobczak.com/inspect/2021/05/15/linux-packages-under-the-hood.html
but I don't see an answer to my question.

Is this your question?

Subject: Re: where are the actual ".deb" packages? all I see are
"pkgcache.bin" and "srcpkgcache.bin"

The answer could be that you are looking in the wrong directory.

Here, I look in the wrong directory for my own *.deb files that APT
archives for me.

$ ls -p /var/cache/apt
archives/ pkgcache.bin srcpkgcache.bin

Instead, this is the correct directory:

/var/cache/apt/archives/

Post by Albretch Mueller
Where are [the .deb files]

The ones kept around after normal package installation/upgrades/etc
are in /var/cache/apt/archives/ .

But the .deb files your homemade script is downloading are in whatever
directory it runs this command

apt --option APT::Architecture="${_ARCH}" --option Dir::Cache="${_DL_DIR}" download "${_DEB_PKG_2DL}"

You think apt is putting them in "${_DL_DIR}". But it isn't.

"${_DL_DIR}" is just where apt is storing its binary cache (as you
requested it to do).

If you want to see the debs you're downloading, list the contents of
the directory you run the apt command in, not "${_DL_DIR}".

Post by Albretch Mueller
or how do I get the ".deb" package out of those binary
files?

You don't. That would be like trying to get your great-grandfather out
of the parish archives of some town he once lived in. His name may be
in some ledger, but you won't find him there.

pkgcache.bin and srcpkgcache.bin are described here in the source you
cite

https://www.juliensobczak.com/inspect/2021/05/15/linux-packages-under-the-hood.html#wynk-apt-cache-files

which in turn cites

APT Cache File Format
http://www.fifi.org/doc/libapt-pkg-doc/cache.html/index.html#contents

But insofar as you only want to know what information those binary
cache files contain, I think studying

$ man apt-cache

would be more profitable use of time.

Here ends my commentary.

Post by Albretch Mueller
_DEB_PKG="gimp"
_ARCH=$(dpkg --print-architecture)
_DT=$(date +%Y%m%d%H%M%S)
_LOG="apt-rdepends_${_DT}_${_DEB_PKG}_${_ARCH}.log"
echo "// __ \$_LOG: |${_LOG}|"
###
time apt-rdepends "${_DEB_PKG}:${_ARCH}" | grep --extended-regexp
'^[a-zA-Z0-9]' | tac > "${_LOG}" 2>&1
ls -l "${_LOG}"
wc -l "${_LOG}"
_DEB_PKG_2DL=$(head -n 1 "${_LOG}")
echo "// __ \$_DEB_PKG_2DL: |${_DEB_PKG_2DL}|"
_DL_DIR="$(pwd)/${_DT}/${_DEB_PKG_2DL}"
mkdir --parents --verbose "${_DL_DIR}"
apt --option APT::Architecture="${_ARCH}" --option
Dir::Cache="${_DL_DIR}" download "${_DEB_PKG_2DL}"
ls -l "${_DL_DIR}"
file "${_DL_DIR}/"*.*
// __ $_LOG: |apt-rdepends_20230226170006_gimp_amd64.log|
-rw-r--r-- 1 user user 3254 Feb 26 17:00
apt-rdepends_20230226170006_gimp_amd64.log
270 apt-rdepends_20230226170006_gimp_amd64.log
// __ $_DEB_PKG_2DL: |xdg-utils|
mkdir: created directory '/home/user/20230226170006'
mkdir: created directory '/home/user/20230226170006/xdg-utils'
Get:1 http://deb.debian.org/debian bullseye/main amd64 xdg-utils all
1.1.3-4.1 [75.5 kB]
Fetched 75.5 kB in 1s (143 kB/s)
total 67616
-rw-r--r-- 1 user user 34678812 Feb 26 17:00 pkgcache.bin
-rw-r--r-- 1 user user 34555088 Feb 26 17:00 srcpkgcache.bin
/home/user/20230226170006/xdg-utils/pkgcache.bin: APT cache data,
version 16.0, little-endian, 97918 packages, 87142 versions
/home/user/20230226170006/xdg-utils/srcpkgcache.bin: APT cache data,
version 16.0, little-endian, 97917 packages, 87141 versions

[Anders Andersson]

Post by davidson

Post by Albretch Mueller
Basically, I am trying to download all packages that are part of the
installation dependencies of a given one into a directory of my
choosing to then install packages on an unexposed machine.

Like Roberto Sanchez wrote, this sounds like a use-case for the
apt-offline package (which I have not used).

For what it's worth, I was very happy with apt-offline last time I
used it but that was a couple of years ago.

It did exactly what I expected and what OP seems to want: It allows
one to keep an offline Debian machine up-to-date and install software
complete with all necessary dependencies.

[Albretch Mueller]

It did exactly what I expected and what OP seems to want ...

Yes, it is, partially. I would like to be able to go into "exposed
mode" (connect to the Internet) using a Debian live DVD and keep my
own work totally off line (you could even use the same machine, I
remove the hard drive and then put it back when not exposed anymore),
but even if the debian utilities: "clone" and "apt-offline" would be
of great help, there is quite a bit of stuff which, of course, is not
debian business to keep.

Say browsers (the best I have found is brave), when you start up a
machine on DL you are once again "virgin" no cookies, no nothing (I
also run mac changer as well), but even though that kind of blank
slate way of saying "hi!" to the Internet is great, you need to know
that you have downloaded a certain file already for which a listing of
the URLs and the path of that data in the owned, unexposed box ...

For the most part, the software needed to build up all the pieces of
that idea exists. There are other aspects which AFAIK need to be
implemented; for example, some sites take ephemerality too seriously,
instead of showing to you a certain URL to the actual data, the may
change the URL or only let you have access to it if you kept a certain
cookie or would let certain js crap run ... but of course you would
have to remove all js from the unexposed data

I am sure that as Lennon used to sing: "I am not the only one" who
has thought of such thing.

lbrtchx

[Albretch Mueller]

by the way regarding that idea, which Linux friendly chipset producer
makes wifi dongles which you can run from a DL start without having to
install anything. it should be detected right away a be good to go.
Would you suggest a particular one?
this very thread started because I recently bought a wifi dongle
which didn't work off of the DL DVD even though the seller advertise
the thing as being "Linux friendly", working on Linux (even though
installing it is a pain where you don't need to have it)

[Andrew M.A. Cater]

Post by Albretch Mueller
by the way regarding that idea, which Linux friendly chipset producer
makes wifi dongles which you can run from a DL start without having to
install anything. it should be detected right away a be good to go.
Would you suggest a particular one?
this very thread started because I recently bought a wifi dongle
which didn't work off of the DL DVD even though the seller advertise
the thing as being "Linux friendly", working on Linux (even though
installing it is a pain where you don't need to have it)

Hi Albretch,

It depends very much on whether the debian-live image includes firmware or
not.

With the next release,(Bookworm/Debian 12), the default installers will
include non-freee firmware at request.

At this point, it is unknown whether anyone will step up to undertake
maintenance for the debian-live media and scripts for the lifecycle
of bookworm. There have been several requests but no confirmed
maintainers.

If there are any left - you might try https://ryf.fsf.org/categories/wireless-adapters - you will be paying a significant premium for this on what amounts to
old, slow chipsets.

All the very best, as ever,

Andy Cater

All the very best, as ever,

All the very best, as ever,

Andy Cater

[Albretch Mueller]

Post by Andrew M.A. Cater
With the next release,(Bookworm/Debian 12), the default installers will
include non-freee firmware at request.
At this point, it is unknown whether anyone will step up to undertake
maintenance for the debian-live media and scripts for the lifecycle
of bookworm. There have been several requests but no confirmed
maintainers.

I try to avoid "political" and "ideological" matters (to call them
something ;-)) to me it would be fine to go dpkg on a few deb packages
when DL is booted up of course without having to connect to the
Internet.

I did search on: "Debian Live" wifi USB -install "boot up"
and got a few hits most of which were not about what I needed. Making
the search a bit more flexible didn't help.