Discussion:
Usage: "debian ... amd64-netinst.iso"
(too old to reply)
John Conover
2024-08-31 03:50:02 UTC
Permalink
What does a "debian ... amd64-netinst.iso" do
with an .iso?

Can it be coverted to a USB. How?

Thanks,

John
--
John Conover, ***@panix.com, http://www.johncon.com/
David Wright
2024-08-31 05:00:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
I plug in a USB stick (which will get completely overwritten),
check from the logs what its device name is (/dev/sdX),
then, as root, execute:
# dd bs=1M if=debian…amd64-netinst.iso of=/dev/sdX
# sync
and wait for the prompt after the sync command.
I then remove the stick and reinsert it, so that the next
step can't cheat and read some cache. I now execute:
# dd bs=1M if=/dev/sdX of=/dev/stdout count=N | sha512sum
where N is the number of blocks written by the first command.
The output should correspond to what's in the SHA512SUMS file
(found in the same directory as the .iso file) if it's a good copy.

The critical step is making sure that you write to the
correct device, of course.

Cheers,
David.
t***@tuxteam.de
2024-08-31 11:40:03 UTC
Permalink
Post by David Wright
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
I plug in a USB stick (which will get completely overwritten),
check from the logs what its device name is (/dev/sdX),
# dd bs=1M if=debian
amd64-netinst.iso of=/dev/sdX
# sync
and wait for the prompt after the sync command.
My recommendation is to add "oflag=sync status=progress" to that "dd"
command above.

Having bothched some sticks with the above method (that "sync" can take
a *long* time with big images).

The "oflag=sync" takes care of syncing each block before proceeding.
The "status=progress" is more of a psychological thing, giving you
an idea of how much has been done without having to kill -USR1 the
dd process. Since my botching has quite probably been pilot error
most of the time, psychological factors might count...

Cheers
--
t
George at Clug
2024-08-31 06:40:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
I used to use cp
# cp my-iso-file.iso /dev/sde   (where the USB was /dev/sde)

However today I used gnome-multi-writer which took a long time but did
the job.

George.
Post by John Conover
     Thanks,
     John
    
--
Thomas Schmitt
2024-08-31 07:10:01 UTC
Permalink
Hi,
What does a "debian ... amd64-netinst.iso" do with an .iso?
Can it be coverted to a USB. How?
I had some difficulties to parse these questions.
Since others read them as question how to make a bootable USB stick
from a netinst ISO image, i add my two cents:

https://wiki.debian.org/XorrisoDdTarget
describes a script which makes sure that the ISO image ends up on the
desired device rather than potentially spoiling your system disk.


Have a nice day :)

Thomas
Max Nikulin
2024-09-02 03:20:01 UTC
Permalink
Post by Thomas Schmitt
https://wiki.debian.org/XorrisoDdTarget
Thomas, are you intentionally linking raw man page instead of a
formatted one
[[DebianMan:xorriso-dd-target|xorriso-dd-target(1)]]?
Is there a reason why the page is not cross-linked with
<https://wiki.debian.org/DebianInstaller/CreateUSBMedia>?

From my point of view this kind of pages should have links to the
Debian install manual and to the Debian CD FAQ in either "Debian
official" block or at least in the "See also" section".
Thomas Schmitt
2024-09-02 06:40:01 UTC
Permalink
Hi,
Post by Max Nikulin
Thomas, are you intentionally linking raw man page instead of a
formatted > one [[DebianMan:xorriso-dd-target|xorriso-dd-target(1)]]?
Once it was intentional, when the package was only in Debian Testing
and no Debian online man page existed.
Now i changed it to your link proposal.
Post by Max Nikulin
Is there a reason why the page is not cross-linked with
<https://wiki.debian.org/DebianInstaller/CreateUSBMedia>?
I will have to study this quite young wiki page ...
Post by Max Nikulin
From my point of view this kind of pages should have links to the
Debian install manual and to the Debian CD FAQ in either
"Debian official" block or at least in the "See also" section".
Do you mean the man page or the wiki page ?

The man page is not specific to Debian.

If you mean the Debian wiki page, can you point me to an example page
where can learn about the style of a "Debian official" block or a
"See also" section ?


Have a nice day :)

Thomas
Thomas Schmitt
2024-09-02 08:10:02 UTC
Permalink
Hi,
Post by Max Nikulin
Post by Thomas Schmitt
https://wiki.debian.org/XorrisoDdTarget
Is there a reason why the page is not cross-linked with
<https://wiki.debian.org/DebianInstaller/CreateUSBMedia>?
After skimming over that page, i see problems:

- It links to https://www.debian.org/devel/debian-installer/
which links to the non-stable daily Debian Testing images
https://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/iso-cd/

- It does not mention that superuser powers are needed for copying
to /dev/sdX and that a wrong substitute for X can spoil a big
spinning disk if present.

- It explains how to format an additional partition by mkdosfs
but not how to create the partition. But that step is the difficult
one, because the partitions in a Debian netinst ISO are so weird
with a reason. I think fdisk is unscrupulous enough. But in
general the hard-to-digest partitions of a Debian amd64 ISO caused
other distros to abandon a few old machines in favor of a more
conventional partition layout in their installation ISOs.

- It proposes to install firmware in the additional partition.
I understand that the netinst ISOs have become so big because they now
contain lots of firmware.

- In its "Method 3" it proposes SYSLINUX as boot loader.
Although it is known to work via EFI from USB stick (other than from
optical media), i have doubts that the proposed command makes the
stick ready for EFI. Quite surely it does not care for Secure Boot.

It would make sense to link from CreateUSBMedia section "Method 1"
to XorrisoDdTarget, maybe even from "Method 2".

But i have few idea how i should motivate a link from XorrisoDdTarget,
which is for users who rarely operate as superuser on raw device files,
to CreateUSBMedia which looks like being for daring testers.
Proposals would be welcome. (Or a courageous wiki editor who feels
entitled to tell the developers what they shall tell the users.)


Have a nice day :)

Thomas
Thomas Schmitt
2024-09-02 10:00:01 UTC
Permalink
Hi,
Post by Thomas Schmitt
- It does not mention that superuser powers are needed for copying
to /dev/sdX and that a wrong substitute for X can spoil a big
spinning disk if present.
I did not pay enough attention to the preamble, which warns of the
danger to spoil a hard disk.
(The warning sign ought to be larger with blinking lights.)

But i still lack a good reason to link from XorrisoDdTarget to
DebianInstaller/CreateUSBMedia.
Post by Thomas Schmitt
From my point of view this kind of pages should have links to the Debian
install manual and to the Debian CD FAQ
The installation manual overlaps with XorrisoDdTarget mainly by
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
but i don't think that this would help much a potential user of
xorriso-dd-target. It proposes what i deem too dangerous for the general
public.

The FAQ
https://www.debian.org/CD/faq/
indeed offers extra information.
I will think about how to point to it and especially to
https://www.debian.org/CD/faq/#verify
which could put more emphasis on verification of USB sticks, especially
in its title
"How can I verify the downloaded ISO images and written optical media?"
I'll think about a proposal which i can submit to debian-cd mailing list.

Maybe i should also mention how to make use of the file /md5sum.txt in
the ISOs.


Have a nice day :)

Thomas
Thomas Schmitt
2024-09-02 11:10:01 UTC
Permalink
Hi,

i added a new section
https://wiki.debian.org/XorrisoDdTarget#How_to_verify_the_result
instead of a mere link because i deem the Debian instructions too
scattered for being suitable for already puzzled and stressed users.

I would appreciate checkreading of this new section by interested
bystanders.


Have a nice day :)

Thomas
Hans
2024-09-02 11:30:01 UTC
Permalink
Post by Thomas Schmitt
Hi,
i added a new section
https://wiki.debian.org/XorrisoDdTarget#How_to_verify_the_result
instead of a mere link because i deem the Debian instructions too
scattered for being suitable for already puzzled and stressed users.
I would appreciate checkreading of this new section by interested
bystanders.
Have a nice day :)
Thomas
Hi Thomas,

nice tutorial, you wrote, very impressiv!

Maybe you might want to add a suggestion for readers, which might be the best
way, to get an ISO to an usb-stick?

The debian manual suggests using the "dd" command and claims, not to use
Rufus, as it might not work.

I myself made good eperiences with dcfldd instead of using dd. In my case i am
creating an ISO-Livefile of a running system with bootcdwrite and then copy it
to an usb-stick using dcfldd. As these are large files (greater than 25GB),
the danger, something gets lost during copy, is high.

And in the past, dd worked not as good as dcfldd. With dd I got more errors.

I already mentioned: dcfldd is testing very copied segment with a hashsum,
that is correct, which dd does not.

As I am doing this builds regularly, I already did about 60-70 copies in the
last, so I think, meanwhile I know, what I am talking about.

A little side effect of this was, that some manufacturers obviously do not
like ext2/3/4 format (to which the usb-stick were reformatted). After about
10th to 15th time of reformate, they died (memory segments got lost). Others I
already during years are reformatted about 50 times or more and are still in
best health.

Anyway, just some ideas and maybe you or someone else finds it usefull.

Best wishes

Hans
e***@gmx.us
2024-09-02 14:20:02 UTC
Permalink
Post by Hans
A little side effect of this was, that some manufacturers obviously do not
like ext2/3/4 format (to which the usb-stick were reformatted). After about
10th to 15th time of reformate, they died (memory segments got lost). Others I
already during years are reformatted about 50 times or more and are still in
best health.
I've had some thumb drives die and that might be the cause. Have you noticed
any patterns?

--
Your pretended fear lest error might step in is like the man who
would keep all wine out of the country lest men should be drunk.
-- Oliver Cromwell
Hans
2024-09-02 15:00:01 UTC
Permalink
Post by e***@gmx.us
I've had some thumb drives die and that might be the cause. Have you noticed
any patterns?
--
Your pretended fear lest error might step in is like the man who
would keep all wine out of the country lest men should be drunk.
-- Oliver Cromwell
No, nothing special. Some IO-errors and one does not boot any more (kernel
crash, due to copy errors, but the ISO itself is ok. Working perfrectly on
another stick and after several copies, this stick is booting, also. However,
too unsecure for my productive systems).

Best

Hans
Thomas Schmitt
2024-09-02 12:10:02 UTC
Permalink
Hi,
Post by Hans
Maybe you might want to add a suggestion for readers, which might be the
best way, to get an ISO to an usb-stick?
That's the actual topic of that wiki page.
Just scroll up and read it from start.
Post by Hans
The debian manual suggests using the "dd" command and claims, not to use
Rufus, as it might not work.
Which manual in particular ?

https://www.debian.org/releases/stable/amd64/ch04s03.en.html
warns of unetbootin, but not of Rufus.
Post by Hans
I myself made good eperiences with dcfldd instead of using dd.
Yes. There are many ways to do the copying. It is the choice of the
target device file that i deem most risky. xorriso-dd-target strives
to make this choice unambiguous by asking the user to newly plug in
the intended USB stick at step 2 of its dialog.
Post by Hans
As these are large files (greater than 25GB),
the danger, something gets lost during copy, is high.
That would be quite a bad USB stick, then.
Normally a good USB stick takes its capacity worth of data without error
or alteration. The new section in XorrisoDdTarget is for making sure
that you don't suffer from the unnormal cases.

How often do you experience bad copy runs ?
Post by Hans
A little side effect of this was, that some manufacturers obviously do not
like ext2/3/4 format (to which the usb-stick were reformatted). After about
10th to 15th time of reformate, they died (memory segments got lost).
A good USB stick's firmware will care for wear leveling, so that
frequently overwritten logical blocks get written to different physical
blocks.
https://en.wikipedia.org/wiki/Wear_leveling


Have a nice day :)

Thomas
Hans
2024-09-02 14:00:02 UTC
Permalink
Hi Thomas,
Post by Thomas Schmitt
That's the actual topic of that wiki page.
Just scroll up and read it from start.
maybe I did not express myself well enough. What I meant is more a desciption,
why xorriso-dd-target is better than dd or dcfldd. Just a little comparision,
which advantages your way got or disadvantages compared to the other.

This does not affect, that your wiki is detailed enough for everyone. Just a
suggestion from me, what I personally would have expected to read. Please note
I say "personally". :)
Post by Thomas Schmitt
Post by Hans
The debian manual suggests using the "dd" command and claims, not to use
Rufus, as it might not work.
You got me! It was not Rufus, it was Unetbootin. I confused both. My fault.
Post by Thomas Schmitt
How often do you experience bad copy runs ?
The last times I would say, every 2nd or 3rd run I got copy errors. Thisis
because the stick itself. I had several from the same manufacturer, all failed
after a while. Other sticks, older and other manufacturer, did not fail at
all. All sticks were 16GB and newly bought. (say: about 1 year old)
Post by Thomas Schmitt
A good USB stick's firmware will care for wear leveling, so that
frequently overwritten logical blocks get written to different physical
blocks.
https://en.wikipedia.org/wiki/Wear_leveling
No no, they are defective. Hardware issue in the bootloader. When reformatting
to vfat, they are still working and (as far as I could ses) no errors.
However, three of my five sticks are completly unusabla. One could no more be
recognized, one only has 128MB writable, and the third one got randomly read/
write errors. Guess, the reformat is not intended for it. But this is crystal
clear: usb-sticks are made for Windows, for Windows and only for Windows. Hey,
are ther other OS in the world???? :)
Post by Thomas Schmitt
Have a nice day :)
Thomas
Have fun!

Hans
Thomas Schmitt
2024-09-02 15:10:01 UTC
Permalink
Hi,
Post by Hans
What I meant is more a desciption,
why xorriso-dd-target is better than dd or dcfldd.
xorriso-dd-target is effectively a frontend to lsblk, umount, and dd.
Insofar i deem dd to be sufficient but not necessarily superior to other
programs which can copy data to a USB stick or memory card.
It's just that dd is nearly always available in a GNU/Linux system.

Originally xorriso-dd-target was only intended as frontend to lsblk
which can evaluate the suitability of target devices for any copy program.
Without any arguments it will just list proposals of device names.

$ xorriso-dd-target
sdc
$

But during conversations with Nio Wiklund alias sudodus, who provides
a tool named "mkusb", it evolved to an optional helper for that program:
https://help.ubuntu.com/community/mkusb#Additional_Feature:_mkusb-plug
and to a quite standalone copier itself.
The sequence of use cases in
https://manpages.debian.org/bookworm/xorriso-dd-target/xorriso-dd-target.1.en.html
reflects this evolution.
While mkusb mostly addresses desktop GUI users, xorriso-dd-target restricts
its dependencies to lsblk and vanilla shell equipment.
Post by Hans
The last times I would say, every 2nd or 3rd run I got copy errors. Thisis
because the stick itself. I had several from the same manufacturer, all failed> after a while. [...] All sticks were 16GB and newly bought. (say: about 1
year old)
That sounds like really bad hardware. Mind to share manufacturer name
and model name ?

(I have a 2 GB Intenso model "USB", a 4 GB SanDisk Cruzer,
a 8 GB Corsair Flash Voyager, and a 128 GB Intenso Ultra Line.
They all are more than 5 years old. I use them rarely, except for the
Corsair, which has a sturdy rubber coating and serves me as encrypted
pocket backup medium.)
Post by Hans
Hardware issue in the bootloader.
You probably mean the firmware, not the bootloader, which would be
data in the payload area of the stick.
Post by Hans
Guess, the reformat is not intended for it. But this is crystal
clear: usb-sticks are made for Windows , for Windows and only for Windows.
It's not _that_ bad. The difference is possibly that FAT filesystems
have less metadata than ext filesystems.
But in that context, nothing is as lightweight on firmware and hardware
as a sequentially written filesystem image. One sweep, no random access,
no repeated writing to a single logical block.

Are you sure that possible write errors to FAT filesystems aren't just
silently ignored by the stick's firmware ?
If you otherwise use it with programs which checkread and complain, then
the impression can appear that only those programs cause errors.
You might get a more realistic impression if you manually checkread and
compare the content of FAT with the data you have written.


Have a nice day :)

Thomas
Hans
2024-09-02 15:30:01 UTC
Permalink
Post by Thomas Schmitt
That sounds like really bad hardware. Mind to share manufacturer name
and model name ?
(I have a 2 GB Intenso model "USB", a 4 GB SanDisk Cruzer,
a 8 GB Corsair Flash Voyager, and a 128 GB Intenso Ultra Line.
They all are more than 5 years old. I use them rarely, except for the
Corsair, which has a sturdy rubber coating and serves me as encrypted
pocket backup medium.)
I believe, it is no problem to share with which usb-sticks I ran into the
problem. The sticks are from Intenso. They are silver, and you can buy it in a
double pack to a very cheap price here in Germany at a well known market place
with a big "M" starting its name.

At the moment I am using "SanDisk Ultra USB 3.0" with 64Gb and had no issues
with these. These are black and made of plastic, shape is tapering to its end
and you can push the contacts out of the case.

At the moment I am testing a new one also from SanDisk. This is itsy bity tiny
(only 3 cm long, 1cm wide and 3mm high) with 128GB. I am using it as my
"prepared-for-the-issue"-stick. I am using YUMI on it (exFAT) and it has now
27 bootable livefilesystems on it (like KALI, GRML, DRBL-Live, clonezilla,
TRK, UBCD4WIN, F4BCD usw. usw.). I think, you can imagine. Among these are
also my own Version of KALI-Life (built with bootcdwrite, about 25GB) and
bigger ones like security-onion (7GB) or TAILS (6,6GB) and smaller ones like
Super-grub-disk, TRK, gparted-live and of course, Debian-Install-iso.
Collection of proven software for many years.

We will see, how well this stick does, as all of these images are regularly
upgraded by me.

Best

Hans
Max Nikulin
2024-09-02 15:20:02 UTC
Permalink
Post by Thomas Schmitt
https://wiki.debian.org/XorrisoDdTarget
Disclaimer: I prefer non-destructive way of making bootable media. For
UEFI it is just copying files, for BIOS it requires some commands like
syslinux and editing of grub configuration. Perhaps I just have not
faced cases when it is not enough. That is why I do not expect that I
will become a regular user of xorriso-dd-target.

If you prefer to keep the wiki page isolated, it is up to you. More
people may discover it if e.g. CategoryDebianInstaller is added. Perhaps
there is a better category.
Post by Thomas Schmitt
Post by Max Nikulin
Thomas, are you intentionally linking raw man page instead of a
formatted > one [[DebianMan:xorriso-dd-target|xorriso-dd-target(1)]]?
Once it was intentional, when the package was only in Debian Testing
and no Debian online man page existed.
<http://manpages.debian.org/testing/xorriso-dd-target>
[[DebianMan:testing/xorriso-dd-target]]
Post by Thomas Schmitt
where can learn about the style of a "Debian official" block or a
"See also" section ?
<https://wiki.debian.org/DebianWiki/EditorGuide#Debian_.22official_material.22_banner>

My point is that the purpose of your script is to enhance the recipe
given in
<https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html>
so this link is suitable to give more context

In my opinion, when a user asks a question similar to the one started
this thread then a link to "Debian GNU/Linux Installation Guide" should
be first answer. This time the XorrisoDdTarget wiki page was posted
earlier, so a link to the installation guide will increase a chance to
make users aware of this document.
Post by Thomas Schmitt
Post by Max Nikulin
Is there a reason why the page is not cross-linked with
<https://wiki.debian.org/DebianInstaller/CreateUSBMedia>?
[...]
Post by Thomas Schmitt
But i have few idea how i should motivate a link from XorrisoDdTarget,
My opinion is that simple "See also" section at the end is enough for
curious users to discover alternatives. I do not consider CreateUSBMedia
as a perfect article, but it has some value. More users notice it means
more chances that it will be improved.
Post by Thomas Schmitt
i added a new section
https://wiki.debian.org/XorrisoDdTarget#How_to_verify_the_result
instead of a mere link because i deem the Debian instructions too
scattered for being suitable for already puzzled and stressed users.
Notice that you have a mention of .sig files in the beginning.

Step by step instruction how to verify a downloaded image is valuable.
Some time ago I was surprised that there is no one in Debian manual and
wiki. However it does not specific to xorriso-dd-target, I would
consider moving content to another article.

Another part is mostly troubleshooting for the case of firmware writing
to EFI/. I can not imagine a case when it may provide something useful
if actual issue is namely writing to a USB block device. Is there an
article on debugging of installer images?

I do not insist on any changes. If you do not like linking to the pages
I asked about, just ignore it.
Franco Martelli
2024-09-02 19:00:01 UTC
Permalink
Post by Thomas Schmitt
Hi,
i added a new section
https://wiki.debian.org/XorrisoDdTarget#How_to_verify_the_result
instead of a mere link because i deem the Debian instructions too
scattered for being suitable for already puzzled and stressed users.
I would appreciate checkreading of this new section by interested
bystanders.
What you wrote in that section it's so searched/wanted by newcomers
that it's a pity that it's published as a section into "XorrisoDdTarget".
Personally I'd like to have that as standalone wiki page in order to
point the wiki link to a newbie that ask for a complete guide to verify
a downloaded ISO image.

Again, it's ugly to have that as a section of "XorrisoDdTarget" wiki page.
So, please, consider to move that to a new wiki page with a title that
sounds like: "Verify authenticity of a Debian downloaded ISO image".

By doing so me and other can easily answer a frequently asked question.
If you agree or not please post your feedback.

Thanks
--
Franco Martelli
Thomas Schmitt
2024-09-02 20:40:01 UTC
Permalink
Hi,
What you wrote in that section it's so searched/wanted by newcomers that
it's a pity that it's published as a section into "XorrisoDdTarget".
Actually it is not the first time that i felt the urge to describe the
SHA and PGP verification:
https://wiki.debian.org/JigdoOnLive#Verify_the_Debian_Live_download
consider to move that to a new wiki page with a title that
sounds like: "Verify authenticity of a Debian downloaded ISO image".
I'm not against that idea.
You could even get a wiki account
https://wiki.debian.org/DebianWiki/EditorQuickStart#Account
and do it on your own.

But i think there are some issues to address:

- How to generally advertise this page ?

- How to name it ?

- How to address all the stuff which is in
https://www.debian.org/CD/faq/#verify
and to which i pointed only briefly ?
I think it is not good to send readers around in the web.
Much better i'd like it all in one text (wearing my noob hat).


So wouldn't it be better to start a petition at debian-cd mailing list
for an augmented https://www.debian.org/CD/faq/#verify ?
To my mind comes:

- Mention "USB keys" already in the section title.

- Replace "Detailed information ... authenticity verification page."
by the last part of my new section: "PGP verification ...".
(The current authenticity verification page is unsuitable for noobs.)

- Add an example like
$ grep -F debian-12.7.0-amd64-netinst.iso SHA512SUMS | sha512sum -c -
debian-12.7.0-amd64-netinst.iso: OK
$
to "Checksums of the downloaded ... "sha256sum"."
(We don't want to show deprecated "fgrep" to noobs. Of course i
continue to use it at home.)

- Add a new section about /md5sum.txt of the ISO.
"How to find the altered files in case of no match"
Derived from my text
"If the verification attempt yields a non-matching checksum ...

- Augment
" $ dd if=<device> count=<sector count> bs=<sector size> | sha512sum
* The computed checksum is to be compared ... SHA256SUMS, etc)."
By an example how to compare the computed checksum with the one in
the SHA512SUMS file.
I currently can only propose
$ grep -F $(dd if=/dev/sdc count=323072 bs=2048 2>/dev/null | \
sha512sum | awk '{print $1}') SHA512SUMS \
|| echo "NO MATCH !"
which would yield
e0bd9ba03084a6fd42413b425a2d20e3731678a31fe5fb2cc84f79332129afca2ad4ec897b4224d6a833afaf28a5d938b0fe5d680983182944162c6825b135ce debian-12.7.0-amd64-netinst.iso
or
NO MATCH !
But the command and the positive answer are quite ugly.
(I could get "OK. MATCH." by "if ... then ... else ... fi" which would
be a nice script but an extra ugly command line.)
Ideas for a more elegant way would be appreciated.

- Change "Optical media verification." to
"Optical media and USB key verification."


Have a nice day :)

Thomas
Max Nikulin
2024-09-03 04:10:01 UTC
Permalink
Post by Thomas Schmitt
Post by Franco Martelli
consider to move that to a new wiki page with a title that
sounds like: "Verify authenticity of a Debian downloaded ISO image".
[...]
Post by Thomas Schmitt
- How to generally advertise this page ?
Link it from articles having similar content or linking
<https://www.debian.org/CD/faq/#verify>: JigdoOnLive, XorrisoDdTarget,
DebianLive, CreateUSBMedia. The last one is referenced in the Debian
install guide, so search engines should discover the new page.
Post by Thomas Schmitt
- How to name it ?
ISOImageVerification, VerifyISOImage, maybe
DebianInstaller/ISOImageVerification. I do not see a kind of portal for
"debian-cd" or images.
Post by Thomas Schmitt
- How to address all the stuff which is in
https://www.debian.org/CD/faq/#verify
Do not try to cover everything exhaustively from the beginning. Just
move the section from XorrisoDdTarget as a starting point.
Post by Thomas Schmitt
So wouldn't it be better to start a petition at debian-cd mailing list
for an augmented https://www.debian.org/CD/faq/#verify ?
I do not read debian-cd, perhaps you know better if it should be done in
advance of after creation of the draft. I think,
<https://www.debian.org/doc/>, <https://www.debian.org/CD>, and wiki
articles should have mutual cross-links and wiki pages should be most
detailed documents. E.g. primary value of
<https://www.debian.org/CD/verify> is fingerprints that can not be
edited by community, however step by step guide is more suitable for wiki.

I recall there was a quite detailed post on debian-user last couple of
years, but I am unsure I can find it. Perhaps

Thomas Schmitt to debian-user. Re: No Public Key. Mon, 14 Nov 2022
09:19:29 +0100.
<https://lists.debian.org/msgid-search/***@scdbackup.webframe.org>

Another candidate is more brief:
Gökşin Akdeniz to debian-user. Re: No Public Key. Sun, 13 Nov 2022
23:46:22 +0300.
<https://lists.debian.org/msgid-search/ad2432d5-0ac7-5212-95e4-***@gmail.com>
john doe
2024-08-31 16:40:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
You have enough data in the iso file to start a Debian installation,
most of the PKGs will be fetched from the internet.
Post by John Conover
Can it be coverted to a USB. How?
cp <ISOFILE> <NAME-OF-USB-DEVICE>.

--
John Doe
Timothy M Butterworth
2024-08-31 22:40:01 UTC
Permalink
Post by john doe
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
A .iso file is a disk image file stored it ISO 9660 file format. An iso is
essentially and image of a CD or DVD. You can burn the iso to a disk very
easily or write it to a USB drive.

You have enough data in the iso file to start a Debian installation,
Post by john doe
most of the PKGs will be fetched from the internet.
Post by John Conover
Can it be coverted to a USB. How?
You can write the iso to a USB drive using the dd command. Optionally you
can use https://etcher.balena.io/ which provides a simple GUI front end.
Post by john doe
cp <ISOFILE> <NAME-OF-USB-DEVICE>.
--
John Doe
--
⢀⣎⠟⠻⢶⣊⠀
⣟⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀
Lee
2024-08-31 18:20:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html

# cp debian.iso /dev/sdX
David Wright
2024-09-01 03:10:02 UTC
Permalink
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.

Cheers,
David.
gene heskett
2024-09-01 04:50:01 UTC
Permalink
Post by David Wright
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
Cheers,
David.
Why should that be difficult? Just do a sha### sum on the device itself.
Before its ever executed. Compare that to the .iso.
Post by David Wright
.
Cheers, Gene Heskett, CET.
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis
Thomas Schmitt
2024-09-01 07:40:01 UTC
Permalink
Hi,
Post by gene heskett
Post by David Wright
The disadvantage of this method is how to check the USB has a good copy.
Why should that be difficult? Just do a sha### sum on the device itself.
The difficulty is that media often are willing to hand out more
bytes than were written when copying the ISO image to them.
Therefore you have to curb reading for checksumming to the size of the
ISO image.
See
https://www.debian.org/CD/faq/#verify
which explains this for optical media like DVD+RW and also for USB sticks.


Have a nice day :)

Thomas
Michael Stone
2024-09-01 13:50:02 UTC
Permalink
Post by t***@tuxteam.de
Post by David Wright
I plug in a USB stick (which will get completely overwritten),
check from the logs what its device name is (/dev/sdX),
# dd bs=1M if=debian…amd64-netinst.iso of=/dev/sdX
# sync
and wait for the prompt after the sync command.
My recommendation is to add "oflag=sync status=progress" to that "dd"
command above.
Having bothched some sticks with the above method (that "sync" can take
a *long* time with big images).
Do *not* use oflag=sync, it will unnecessarily slow things down with no
advantage. Instead, if you really want to make sure things make it to
disk, use oflag=fdatasync, which will prevent dd from exiting until
everything has been written to disk, but without writing blocks one at a
time.

In general the recommended "cp" mechanism (followed by sync) is better
for most uses as it's a much simpler command line and much harder to
screw up with typos. dd has quite a lot of cargo cult voodoo around it,
but it's the best tool for a very small and specialized set of cases.
(It's in the muscle memory of a lot of older users, but that's no
reason we need to impose it on new users.)
Post by t***@tuxteam.de
Post by David Wright
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
I don't understand why it would be any harder or easier to check that
there's a good copy with cp vs some other tool, so this seems like
strange advice.
David Wright
2024-09-01 15:00:01 UTC
Permalink
Post by Michael Stone
Post by David Wright
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
I don't understand why it would be any harder or easier to check that
there's a good copy with cp vs some other tool, so this seems like
strange advice.
So your command line would read …

# … …

Cheers,
David.
Hans
2024-09-01 15:30:01 UTC
Permalink
Post by David Wright
So your command line would read …
# … …
Cheers,
David.
I believe, instead of using dd for copying to the usb-stick, one might want to
use dcfldd for it.

dcfldd is en enhanced version of dd with hashsum check during copy. It was
created for forensic matters, where no data may have gone lost.

Best

Hans
David Wright
2024-09-01 17:10:01 UTC
Permalink
Post by Hans
Post by David Wright
So your command line would read …
# … …
I believe, instead of using dd for copying to the usb-stick, one might want to
use dcfldd for it.
dcfldd is en enhanced version of dd with hashsum check during copy. It was
created for forensic matters, where no data may have gone lost.
Same response: please write me a command line, but in this case
for writing the stick rather than just checking it.

# … …

Cheers,
David.
Felix Miata
2024-09-01 17:30:02 UTC
Permalink
Post by David Wright
Post by Hans
Post by David Wright
So your command line would read …
# … …
I believe, instead of using dd for copying to the usb-stick, one might want to
use dcfldd for it.
dcfldd is en enhanced version of dd with hashsum check during copy. It was
created for forensic matters, where no data may have gone lost.
Same response: please write me a command line, but in this case
for writing the stick rather than just checking it.
# /usr/bin/ddrescue amd64-netinst.iso /dev/sdj

# apt-get install gddrescue
...
# which ddrescue
/usr/bin/ddrescue
# apt-cache show gddrescue
...
Description-en: GNU data recovery tool
The gddrescue tool copies data from one file or block device
(hard disc, cdrom, etc) to another, trying hard to rescue data
in case of read errors.
...
#

ddrescue automatically provides nice status reporting while it works.
--
Evolution as taught in public schools is, like religion,
based on faith, not based on science.

Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata
Michael Stone
2024-09-02 15:30:01 UTC
Permalink
Post by Hans
Post by David Wright
So your command line would read …
# … …
Cheers,
David.
I believe, instead of using dd for copying to the usb-stick, one might want to
use dcfldd for it.
dcfldd is en enhanced version of dd with hashsum check during copy. It was
created for forensic matters, where no data may have gone lost.
no, you should simply verify the checksum after you write it if you
really want to waste time checking things. (the instructions include a
reference to a handy script to do this.) I think it's mostly a waste of
time as the installer validates packages as they're read from the drive.
there's a small chance that the initial stages of the process could be
messed up (the parts that do the validation) but the odds that they're
both corrupt and working well enough to run and validate the later
stages are low enough that I personally am not going to waste time
worrying about it.
Lee
2024-09-01 18:20:01 UTC
Permalink
Post by David Wright
Post by Michael Stone
Post by David Wright
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
I never worried about it. Yeah, I know... Bad Lee [slaps wrist]
Post by David Wright
Post by Michael Stone
I don't understand why it would be any harder or easier to check that
there's a good copy with cp vs some other tool, so this seems like
strange advice.
So your command line would read …
Same as what's in the FAQ[1] .. and then I'd spin my wheels trying to
figure out what's wrong and not too much later say f*kkit and boot off
my just written USB drive.

***@i3668 /cygdrive/c/temp/debian
$ cp debian-12.7.0-amd64-netinst.iso /dev/sdb

***@i3668 /cygdrive/c/temp/debian
$ sync

***@i3668 /cygdrive/c/temp/debian
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso /dev/sdb
Piping 323072 blocks of '/dev/sdb' through 'sha512sum'
to verify checksum list item 'debian-12.7.0-amd64-netinst.iso'.
323072+0 records in
323072+0 records out
661651456 bytes (662 MB, 631 MiB) copied, 7.11874 s, 92.9 MB/s
Found: 9a165064baabd7ffb7063f02136e001a2c78c85c8b4a1ae97dfd81abd5641b32c6f04f4bdcd26d69bf6d78b1589bf2dcf75895106ae9e7794441fd2fc10dbe57
Expected: e0bd9ba03084a6fd42413b425a2d20e3731678a31fe5fb2cc84f79332129afca2ad4ec897b4224d6a833afaf28a5d938b0fe5d680983182944162c6825b135ce
MISMATCH: '/dev/sdb' checksum differs from
'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'

***@i3668 /cygdrive/c/temp/debian
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso
Piping 323072 blocks of 'debian-12.7.0-amd64-netinst.iso' through 'sha512sum'
to verify checksum list item 'debian-12.7.0-amd64-netinst.iso'.
323072+0 records in
323072+0 records out
661651456 bytes (662 MB, 631 MiB) copied, 2.65785 s, 249 MB/s
Ok: 'debian-12.7.0-amd64-netinst.iso' matches
'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'

Regards,
Lee

[1] https://www.debian.org/CD/faq/#verify
Thomas Schmitt
2024-09-01 19:30:01 UTC
Permalink
Hi,
[...] I'd spin my wheels trying to
figure out what's wrong and not too much later say f*kkit and boot off
my just written USB drive.
[...]
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso /dev/sdb
...
661651456 bytes (662 MB, 631 MiB) copied, 7.11874 s, 92.9 MB/s
MISMATCH: '/dev/sdb' checksum differs from 'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'
[...]
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso
...
661651456 bytes (662 MB, 631 MiB) copied, 2.65785 s, 249 MB/s
Ok: 'debian-12.7.0-amd64-netinst.iso' matches 'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'
The script correctly truncated the data stream from /dev/sdb to the
size that it read from the superblock of its ISO filesystem.
If the checksums do not match, then some bytes in that range on /dev/sdb
are not as they are in debian-12.7.0-amd64-netinst.iso.


If you are interested in learning where, please keep the stick as it is
now. Especially do not mount its EFI partition and do not plug it into
a running MS-Windows system, which would happily alter bytes in that
partition.

The ISO contains a file /md5sum.txt with MD5 checksums for most of the
files in the ISO. Although MD5 is nowadays easy to counterfei, it still is
a useful transport checksum.

For using that file, i choose two file names which suit me:

mountpoint=/mnt/iso
report_file=/tmp/md5_report

and mount the ISO filesystem. You would probably do:

sudo mount /dev/sdb "$mountpoint"

I let md5sum verify the listed files

cd "$mountpoint"
md5sum -c ./md5sum.txt >"$report_file" 2>&1

Finally i look into "$report_file" by a text editor

view "$report_file"

I search for lines which end by the text snipped ": FAILED".

My report file from debian-12.6.0-amd64-netinst.iso instead of /dev/sdb
shows one failure, to my surprise:
./boot/grub/efi.img: FAILED
That's the file which holds the EFI partition. I must have played with it.
The sha256sum result with the .iso file does not match the one in
https://cdimage.debian.org/mirror/cdimage/archive/12.6.0/amd64/iso-cd/SHA256SUMS
any more.

So i download debian-12.7.0-amd64-netinst.iso, check its SHA256,
and repeat above check procedure.
This time no lines with ": FAILED" at their end.


Have a nice day :)

Thomas
Lee
2024-09-02 06:10:01 UTC
Permalink
Post by Thomas Schmitt
Hi,
[...] I'd spin my wheels trying to
figure out what's wrong and not too much later say f*kkit and boot off
my just written USB drive.
[...]
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso /dev/sdb
...
661651456 bytes (662 MB, 631 MiB) copied, 7.11874 s, 92.9 MB/s
MISMATCH: '/dev/sdb' checksum differs from 'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'
[...]
$ check_debian_iso SHA512SUMS debian-12.7.0-amd64-netinst.iso
...
661651456 bytes (662 MB, 631 MiB) copied, 2.65785 s, 249 MB/s
Ok: 'debian-12.7.0-amd64-netinst.iso' matches 'debian-12.7.0-amd64-netinst.iso' in 'SHA512SUMS'
The script correctly truncated the data stream from /dev/sdb to the
size that it read from the superblock of its ISO filesystem.
If the checksums do not match, then some bytes in that range on /dev/sdb
are not as they are in debian-12.7.0-amd64-netinst.iso.
If you are interested in learning where, please keep the stick as it is
now. Especially do not mount its EFI partition and do not plug it into
a running MS-Windows system, which would happily alter bytes in that
partition.
Oops.. I wrote to the stick using the Cygwin cp on an MS-Windows
machine, so I'm guessing the damage was done even before ejecting the
stick. But I'm more interested in what went wrong than exactly where
the write/verification failed.

I tried everything all over again on my Debian machine and .. it all works :)
as expected even!!

so yeah, the obvious conclusion is that "what went wrong" is that I
used a Windows machine. *sigh*

As a very low priority, how was I able to install Debian on my laptop
and have everything work when I did the exact same thing for my
laptop?
download the .iso on the windows machine
cp the iso to /dev/sdb using cygwin on the windows machine
boot the laptop from the usb stick and install debian

My June "how2 format a flash drive" msg was after installing Debian on
the laptop.

Lee
Thomas Schmitt
2024-09-02 07:00:02 UTC
Permalink
Hi,
Post by Lee
Oops.. I wrote to the stick using the Cygwin cp on an MS-Windows
machine, so I'm guessing the damage was done even before ejecting the
stick.
MS-Windows can eject a stick ?
xorriso silently fails to do so:

xorriso -outdev stdio:/dev/sdc -eject out

(Sorry i could not refrain from this nonsense :))
Post by Lee
But I'm more interested in what went wrong than exactly where
the write/verification failed.
IIRC i once learned from a report about a "FAILED" md5sum.txt check
that it was the EFI partition which got altered and that various
proprietary software companies feel entitled to add (rather harmless)
files to any FAT filesystem which their software can see.
Post by Lee
so yeah, the obvious conclusion is that "what went wrong" is that I
used a Windows machine. *sigh*
Well, you have to expect such things to happen there.
They wipe your bottom and your nose ... using the same cloth.
Post by Lee
As a very low priority, how was I able to install Debian on my laptop
and have everything work when I did the exact same thing for my
laptop?
Maybe this time you pulled out the stick and put it back in while
MS-Windows was still running ?
I have no tangible idea other than to propose to do some experiments.


Have a nice day :)

Thomas
Lee
2024-09-02 17:00:01 UTC
Permalink
Post by Thomas Schmitt
Hi,
Post by Lee
Oops.. I wrote to the stick using the Cygwin cp on an MS-Windows
machine, so I'm guessing the damage was done even before ejecting the
stick.
MS-Windows can eject a stick ?
xorriso -outdev stdio:/dev/sdc -eject out
(Sorry i could not refrain from this nonsense :))
but it isn't nonsense. Welcome to the world of Windowz, where one
'ejects' a USB stick and then gets a pop-up saying something about
safe to remove the hardware now.

Maybe there's a command to unmount / sync / whatever a USB stick but
I've always used Windows Explorer and the only option for what to do
before unplugging a USB stick is to eject it.
Post by Thomas Schmitt
Post by Lee
But I'm more interested in what went wrong than exactly where
the write/verification failed.
IIRC i once learned from a report about a "FAILED" md5sum.txt check
that it was the EFI partition which got altered and that various
proprietary software companies feel entitled to add (rather harmless)
files to any FAT filesystem which their software can see.
Post by Lee
so yeah, the obvious conclusion is that "what went wrong" is that I
used a Windows machine. *sigh*
Well, you have to expect such things to happen there.
Well, no.. I used to have bash scripts that worked on SunOS and then
later Red Hat as well as cygwin. I've gotten used to things Just
Working across platforms, so this failing is a bit of a surprise.
Post by Thomas Schmitt
They wipe your bottom and your nose ... using the same cloth.
They certainly are getting more .. abusive parental, thinking they
know better than you and doing whatever regardless of what you want.
Post by Thomas Schmitt
Post by Lee
As a very low priority, how was I able to install Debian on my laptop
and have everything work when I did the exact same thing for my
laptop?
Maybe this time you pulled out the stick and put it back in while
MS-Windows was still running ?
I have no tangible idea other than to propose to do some experiments.
Me either. I'm not going to worry about it - other than reinstall
Debian on my laptop from a USB stick written on a Debian machine :)

Regards
Lee
The Wanderer
2024-09-02 17:20:01 UTC
Permalink
Post by Lee
Post by Thomas Schmitt
Hi,
Post by Lee
Oops.. I wrote to the stick using the Cygwin cp on an
MS-Windows machine, so I'm guessing the damage was done even
before ejecting the stick.
MS-Windows can eject a stick ?
xorriso -outdev stdio:/dev/sdc -eject out
(Sorry i could not refrain from this nonsense :))
but it isn't nonsense. Welcome to the world of Windowz, where one
'ejects' a USB stick and then gets a pop-up saying something about
safe to remove the hardware now.
Maybe there's a command to unmount / sync / whatever a USB stick but
I've always used Windows Explorer and the only option for what to do
before unplugging a USB stick is to eject it.
My understanding is that when you tell Windows to "eject" removable
media, it does whatever is necessary to prepare that media for clean
removal.

With an optical disc, this includes unmounting the filesystem (and any
necessary cleanup related to the drive letter), and also includes
sending a physical eject command to the optical drive.

With a USB flash drive, it includes unmounting the filesystem, which
likely also includes an implicit sync operation.

If a new type of removable media comes along in the future, they'll
probably preserve the same prepare-for-safe-removal interface, and make
it do whatever the new type may need under the hood.

I have always inferred that the only reason for the "eject" terminology
is that the interface and terminology were established during a period
before USB flash drives existed, when optical drives were the only
removable media (other than floppy disks, which I think were already
largely obsolete by the time this interface came along).

I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
--
The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw
Michael Stone
2024-09-02 17:50:01 UTC
Permalink
Post by The Wanderer
I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
Yes. Windows actually does have an option to eliminate the eject step by
performing all writes synchronously (I think it may even be the default
in current versions) but it makes working with removable media painfully
slow. Linux has a similar option (mount with the "sync" option) but it
is generally not used because of the really significant performance
impact, and because it can cause much faster wear on flash devices
(because there is no opportunity to coalesece small writes into larger
writes).
t***@tuxteam.de
2024-09-02 18:40:01 UTC
Permalink
Post by Michael Stone
Post by The Wanderer
I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
Yes. Windows actually does have an option to eliminate the eject step by
performing all writes synchronously (I think it may even be the default in
current versions) but it makes working with removable media painfully slow.
Linux has a similar option (mount with the "sync" option) but it is
generally not used because of the really significant performance impact, and
because it can cause much faster wear on flash devices (because there is no
opportunity to coalesece small writes into larger writes).
This is exactly why I'm always trying to "sell" the "oflag=sync" option of
dd. You are going to write each block exactly once, then plop the medium
out. So whithout, it'll take you 10 sec, with it'll take you as much. But
it's more transparent with, because without, the OS is flushing cache in
the background.

I know, I repeat myself:

dd if=/path/to/image of=/path/to/device bs=1M oflag=sync status=progress

In my experience, block sizes between a couple of hundred K and a couple
of M do well on flash drives. The default of 512 is very bad, and above
a couple of M it starts to degrade again. It's a very "flat" maximum.
YMMV on that.

Cheers
--
t
Michael Stone
2024-09-05 00:40:01 UTC
Permalink
Post by t***@tuxteam.de
Post by Michael Stone
Post by The Wanderer
I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
Yes. Windows actually does have an option to eliminate the eject step by
performing all writes synchronously (I think it may even be the default in
current versions) but it makes working with removable media painfully slow.
Linux has a similar option (mount with the "sync" option) but it is
generally not used because of the really significant performance impact, and
because it can cause much faster wear on flash devices (because there is no
opportunity to coalesece small writes into larger writes).
This is exactly why I'm always trying to "sell" the "oflag=sync" option of
dd. You are going to write each block exactly once, then plop the medium
out. So whithout, it'll take you 10 sec, with it'll take you as much. But
it's more transparent with, because without, the OS is flushing cache in
the background.
It's a waste of time but if you make it happy you do you. Please don't
impose it on others.
t***@tuxteam.de
2024-09-05 04:50:01 UTC
Permalink
[...]
Post by Michael Stone
Post by t***@tuxteam.de
This is exactly why I'm always trying to "sell" the "oflag=sync" option of
dd [...]
It's a waste of time but if you make it happy you do you. Please don't
impose it on others.
That's a strange thing to say.

I don't have any power to impose oflag=sync on anyone (and if I had, I wouldn't
use it for such a thing: why should I?)

BTW, my reason for using it is not that it's a "waste of time" otherwise, but
rather that waiting on an (unpredictable) sync to finish feels less comfy than
to have a rough idea of how long it'll take (thanks to status=progress).

Cheers
--
t
Thomas Schmitt
2024-09-05 09:30:01 UTC
Permalink
Hi,
Post by t***@tuxteam.de
This is exactly why I'm always trying to "sell" the "oflag=sync" option of
dd. You are going to write each block exactly once, then plop the medium
out. So whithout, it'll take you 10 sec, with it'll take you as much. But
it's more transparent with, because without, the OS is flushing cache in
the background.
I let xorriso-dd-target use
bs=1M oflag=dsync
with the dd run which copies the ISO image. Not only to minimize the time
of the final sync command but also to get realistic numbers from
status=progress
which elsewise reports fantastic write speeds in the beginning and then
looks quite erratic when the kernel decides to write data to the USB stick
before buffering more of the data sent by dd.

What i experienced as puzzling in the documentation was the difference
between oflag=dsync and oflag=sync (not to be confused with conv=sync).
The info document of dd is more verbous than the man page but still not
really helpful in this aspect:

‘dsync’
Use synchronized I/O for data. For the output file, this
forces a physical write of output data on each write. For the
input file, this flag can matter when reading from a remote
file that has been written to synchronously by some other
process. Metadata (e.g., last-access and last-modified time)
is not necessarily synchronized.

‘sync’
Use synchronized I/O for both data and metadata.

I understand from
https://sources.debian.org/src/coreutils/9.4-3.1/src/dd.c
and
https://sources.debian.org/src/coreutils/9.4-3.1/lib/fd-reopen.c
that the difference is the one between open(2) flags O_SYNC and O_DSYNC.
man 2 open says:

To understand the difference between the two types of completion, con‐
sider two pieces of file metadata: the file last modification timestamp
(st_mtime) and the file length. All write operations will update the
last file modification timestamp, but only writes that add data to the
end of the file will change the file length. The last modification
timestamp is not needed to ensure that a read completes successfully,
but the file length is. Thus, O_DSYNC would only guarantee to flush
updates to the file length metadata (whereas O_SYNC would also always
flush the last modification timestamp metadata).

When writing to a device file, neither its length nor timestamp are of
importance for the resulting state of the USB stick. Length does not
change during writing and the timestamp vanishes with the device file
when the stick gets unplugged.
Nevertheless i add a standalone sync command to the last dd run of
xorriso-dd-target, just to care for any other remaining buffered data
e.g. from zeroizing the potential GPT backup header at the end of the
USB stick.
Post by t***@tuxteam.de
It's a waste of time
My experience with or without bs=1M oflag=dsync is like the sniffles:
They last seven days with a doctor and a week without.


Have a nice day :)

Thomas
t***@tuxteam.de
2024-09-05 10:00:01 UTC
Permalink
Post by Thomas Schmitt
Hi,
Post by t***@tuxteam.de
This is exactly why I'm always trying to "sell" the "oflag=sync" option of
dd. You are going to write each block exactly once, then plop the medium
out. So whithout, it'll take you 10 sec, with it'll take you as much. But
it's more transparent with, because without, the OS is flushing cache in
the background.
I let xorriso-dd-target use
bs=1M oflag=dsync
with the dd run which copies the ISO image. Not only to minimize the time
of the final sync command but also to get realistic numbers from
status=progress
which elsewise reports fantastic write speeds in the beginning and then
looks quite erratic when the kernel decides to write data to the USB stick
before buffering more of the data sent by dd.
What i experienced as puzzling in the documentation was the difference
between oflag=dsync and oflag=sync (not to be confused with conv=sync).
The info document of dd is more verbous than the man page but still not
‘dsync’
Use synchronized I/O for data [...]
‘sync’
Use synchronized I/O for both data and metadata.
[...]
Post by Thomas Schmitt
When writing to a device file, neither its length nor timestamp are of
importance for the resulting state of the USB stick [...]
That was my take, too: in the "USB stick" case, there's practically no
difference. And when writing to a file, I'll happily take the hit of
updating the metadata too for consistent results.

So I decided to burden my dwindling memory with one letter less :-)
Post by Thomas Schmitt
change during writing and the timestamp vanishes with the device file
when the stick gets unplugged.
Nevertheless i add a standalone sync command to the last dd run of
xorriso-dd-target, just to care for any other remaining buffered data
e.g. from zeroizing the potential GPT backup header at the end of the
USB stick.
You know far more about those innards than I can hope too :-)
Post by Thomas Schmitt
Post by t***@tuxteam.de
It's a waste of time
They last seven days with a doctor and a week without.
The sticks I've used (and the CPU and what not) are *much* faster with
bs=1M than with the default (512, with bs). The oflag=sync is more about
predictability, as you write above.

Cheers
--
t
Max Nikulin
2024-09-06 03:00:01 UTC
Permalink
Post by Thomas Schmitt
I let xorriso-dd-target use
bs=1M oflag=dsync
May too small bs value cause write multiplication if internal flash
erasure block size is much larger? I have seen claims that it can be
e.g. 12M
<https://superuser.com/questions/673847/if-i-align-a-partition-to-24mib-then-is-it-also-aligned-to-any-divider-of-this-s>
I have an impression that actual value is not exposed, so it is unknown
to kernel. My concern is wearing, I have no idea if firmware may perform
partial overwrites without apparent impact on speed.

Are you against "sync" command because it syncs all drives, not the
specific one (besides smooth progress report)? If USB mass storage
driver does not allow to turn port power off before write completion
then it might be a workaround. From my point of view, simple cp
suggested by the install guide is quite reasonable in comparison to dd.

Another my question is concerning reading of media and -x argument of
isosize. Is it really necessary? I can not figure out what corner cases
are not covered by "head -c BYTES /dev/sdc".
Jeffrey Walton
2024-09-06 03:20:02 UTC
Permalink
Post by Max Nikulin
Post by Thomas Schmitt
I let xorriso-dd-target use
bs=1M oflag=dsync
May too small bs value cause write multiplication if internal flash
erasure block size is much larger? I have seen claims that it can be
e.g. 12M
<https://superuser.com/questions/673847/if-i-align-a-partition-to-24mib-then-is-it-also-aligned-to-any-divider-of-this-s>
I have an impression that actual value is not exposed, so it is unknown
to kernel. My concern is wearing, I have no idea if firmware may perform
partial overwrites without apparent impact on speed.
I've run into this situation (more than once)... `dd` fails to write a
bootable image with a block size of 1M. But using a block size of 512
results in a bootable image. It used to happen regularly on arm dev
boards, like BeagleBoards and CubieTrucks and Wandboards. I don't
recall it happening lately.

I prefer the larger block size because the image writes faster. But if
the board fails to boot, I drop back to 512 to (re)write the image.

And I always finish the `dd` command with a `&& sync`.
Post by Max Nikulin
Are you against "sync" command because it syncs all drives, not the
specific one (besides smooth progress report)? If USB mass storage
driver does not allow to turn port power off before write completion
then it might be a workaround. From my point of view, simple cp
suggested by the install guide is quite reasonable in comparison to dd.
Another my question is concerning reading of media and -x argument of
isosize. Is it really necessary? I can not figure out what corner cases
are not covered by "head -c BYTES /dev/sdc".
Jeff
Thomas Schmitt
2024-09-06 08:00:01 UTC
Permalink
Hi,
Post by Thomas Schmitt
I let xorriso-dd-target use
bs=1M oflag=dsync
May too small bs value cause write multiplication if internal flash erasure
block size is much larger? [...] My concern is wearing,
Is there evidence that sync size a smaller than such an internal size
is harmful ?

Writing small chunks would also happen with any scattered writing with
smaller chunks, which would happen when medium sized files get written
here and there in the (FAT) filesystem.
Firmware could try to accumulate data in finer granulated memory as long
as the data would go to the same physical chunk, and only later write
to the coarsly granulated memory.
Sequential writing by dd would well fit into such a caching strategy.
I've run into this situation (more than once)... `dd` fails to write a
bootable image with a block size of 1M. But using a block size of 512
results in a bootable image. It used to happen regularly on arm dev
boards, like BeagleBoards and CubieTrucks and Wandboards.
I would be interested in more details (error messages ? no booting ?).
I don't recall it happening lately.
If it happens again, please report.

If ever, bs=512 would invite for more extra block erasures than bs=1M.
I have no idea if firmware may perform
partial overwrites without apparent impact on speed.
Is there evidence ?
I know from own experiments that very small sizes like bs=512 cause slow
copying with and without oflag=dsync. Also very large sizes yielded less
throughput than bs=1M. (My sample set of USB sticks used is small.)
Are you against "sync" command because it syncs all drives,
I understand dd oflag=sync only applies to the target file or device and
that the difference to oflag=dsync is about metadata of the target file
like its timestamps.

But xorriso-dd-target runs sync(1) in the end. Just to be sure.
(Sorry for any other data which get onto storage device early.)
From my point of view, simple cp suggested by the install guide
is quite reasonable in comparison to dd.
cp is not suitable for erasing the last block of the USB stick where the
header of the backup GPT may reside, if the stick was GPT partitioned.
cp does not display pacifier messages and may end substantially earlier
than the write process from kernel to USB stick.
Another my question is concerning reading of media and -x argument of
isosize. Is it really necessary? I can not figure out what corner cases
are not covered by "head -c BYTES /dev/sdc".
If you put emphasis on avoiding dd, then this would be an alternative.


Have a nice day :)

Thomas
Max Nikulin
2024-09-06 12:00:01 UTC
Permalink
Post by Thomas Schmitt
Post by Jeffrey Walton
I've run into this situation (more than once)... `dd` fails to write a
bootable image with a block size of 1M. But using a block size of 512
results in a bootable image. It used to happen regularly on arm dev
boards, like BeagleBoards and CubieTrucks and Wandboards.
I would be interested in more details (error messages ? no booting ?).
My expectation is that "boot" is not really relevant here. Verification
of written data should fail. The question if it is an issue with flash
firmware or with a board where dd was running. Comparison of original
image and data read back may reveal some pattern of corrupted bytes.
Another variant that flash firmware could not tell the host to wait till
a portion of data was actually written.

I would be really surprised if reading resulted in exactly same data,
attempt to boot from the device failed.
Max Nikulin
2024-09-07 03:50:01 UTC
Permalink
Post by Thomas Schmitt
Post by Thomas Schmitt
I let xorriso-dd-target use
bs=1M oflag=dsync
May too small bs value cause write multiplication if internal flash erasure
block size is much larger? [...] My concern is wearing,
Is there evidence that sync size a smaller than such an internal size
is harmful ?
It is no more than my guess. My expectation that some flash firmware may
respect explicit syncs.
Post by Thomas Schmitt
Sequential writing by dd would well fit into such a caching strategy.
...unless explicit syncs force writing received data. Sequential writing
with frequent syncs should not be an issue (besides some buggy devices)
if the drive has been fully erased in advance. I am unaware how to erase
a USB pendrive or a SD card. Perhaps some vendors provide special tools.

My hypothesis is that writing to 8MiB erase block in 1MiB chunks over
old data may result in
- erasure or next free segment
- write 0 new data, 1-7 old data
- erasure or next free segment
- write 0-1 new data, 2-7 old data
...
- erasure or next free segment
- write 0-6 new data, 7 old data
- erasure or next free segment
- write 0-7 new data
Of course, firmware may react on syncs in a different way.
Post by Thomas Schmitt
I have no idea if firmware may perform
partial overwrites without apparent impact on speed.
Is there evidence ?
No.
Post by Thomas Schmitt
I know from own experiments that very small sizes like bs=512 cause slow
copying with and without oflag=dsync. Also very large sizes yielded less
throughput than bs=1M. (My sample set of USB sticks used is small.)
I have only a few USB sticks and SD cards as well and I use some of them
for backup, some are rather old. That is why I asking questions rather
than making statements.

Are there confirmed cases when dd with fixed bs works faster (or more
reliable) than cp built-in strategy? Of course, it better to have some
evidence that dd bs=1M may cause negative impact as well.

Due to my preference to non-destructive way of making bootable media, I
unlikely will be a source of data points. I noticed that people were
arguing concerning oflag=sync so I expected that they may provide more info.
Post by Thomas Schmitt
From my point of view, simple cp suggested by the install guide
is quite reasonable in comparison to dd.
cp is not suitable for erasing the last block of the USB stick where the
header of the backup GPT may reside, if the stick was GPT partitioned.
It is a perfectly valid case of dd usage. However I do not expect that
many users have GPT partition table on USB pendrives, so cp should be
suitable for the install guide. On the other hand, some may forget about
their experiments, so the additional measure is suitable for scripts and
for troubleshooting docs.

e***@gmx.us
2024-09-02 19:10:01 UTC
Permalink
Post by The Wanderer
I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
Thunar has "Safely Remove", which not only does sync && umount, it also
makes the partition disappear from its listing. It (but not the drive) also
disappears from lsblk's output. I'm not sure what happens if there's >1
mountable partition on the drive. Must investigate further.

--
And we never failed to fail / It was the easiest thing to do -- CSN
Max Nikulin
2024-09-05 03:40:01 UTC
Permalink
Post by The Wanderer
Post by Lee
Post by Thomas Schmitt
MS-Windows can eject a stick ?
[...]
Post by The Wanderer
Post by Lee
Post by Thomas Schmitt
(Sorry i could not refrain from this nonsense :))
but it isn't nonsense. Welcome to the world of Windowz, where one
'ejects' a USB stick and then gets a pop-up saying something about
safe to remove the hardware now.
[...]
Post by The Wanderer
My understanding is that when you tell Windows to "eject" removable
media, it does whatever is necessary to prepare that media for clean
removal.
GNOME uses "eject" as well:
<https://help.gnome.org/users/gnome-help/stable/files-removedrive.html.en>
So users should be familiar with the word in this particular context.
Post by The Wanderer
I have always treated the *nix equivalent to "eject", for the purpose of
a USB flash drive, as being 'umount /path/to/mount/location' - which, if
I'm not mistaken, does include an implicit sync operation.
Dolphin (KDE) "Safely remove" additionally switches off power on the USB
port. In the case of CLI it is

udisksctl power-off -b /dev/sdb

(or uhubctl, direct writing to /sys/bus/usb/devices/**/power, etc.)

I find it a reasonable measure in the case of a spinning external HDD to
ensure that it is stopped. Another rule is to unplug it from the laptop
USB port at first (without touching HDD) and only then detach the cable
from the HDD case.
Michael Stone
2024-09-02 15:20:02 UTC
Permalink
Post by David Wright
Post by Michael Stone
Post by David Wright
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
I don't understand why it would be any harder or easier to check that
there's a good copy with cp vs some other tool, so this seems like
strange advice.
So your command line would read …
What are you even going on about?
David Wright
2024-09-03 05:10:01 UTC
Permalink
Post by Michael Stone
Post by David Wright
Post by Michael Stone
Post by David Wright
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
The disadvantage of this method is how to check the USB has a good copy.
I don't understand why it would be any harder or easier to check that
there's a good copy with cp vs some other tool, so this seems like
strange advice.
So your command line would read …
What are you even going on about?
Well, I might expect you to use a command something like:

# dd bs=1M if=/dev/sdX of=/dev/stdout count=N | sha512sum

to check for a good copy. And yet you seem awfully keen to
avoid using dd because of its alleged "cargo cult voodoo",
whatever that is, and because apparently you can screw it up
with typos (what, in "dd", "bs", "if" and "of"?). But about the
only method I've seen to check the copy is good is to use dd.

If you've copied the file with dd, you don't even have to type
half of the checking command because of command history. And
you don't have to calculate the count of blocks to check
because dd has just told you that.

That's leaving aside the fact that if copying (or checking) is
taking a suspiciously long time, you can ascertain dd's progress
even when you didn't ask for progress in the command line.

Like you, I can't see the point of using dcfldd. Its man page
advertises that it can verify the target drive, but that doesn't
work for this particular application. And ddrescue seems to be
another program that's pointless for this use. As for cp, I don't
see any particular benefit, and a disadvantage is that it's mute.

Sorry if I tried your patience. (BTW I am subscribed.)

Cheers,
David.
d***@howorth.org.uk
2024-09-01 12:20:01 UTC
Permalink
Post by Lee
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
https://www.debian.org/releases/bookworm/amd64/ch04s03.en.html
# cp debian.iso /dev/sdX
You missed the next line

# sync

It's there for a reason.
Joe B
2024-08-31 21:10:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Can it be coverted to a USB. How?
A netinst iso is just a smaller iso to install Debian. It doesn't have the full desktop or everything the DVD and regular iso has. You have to be connected to the Internet during the whole install

Burn it to USB and boot the usb

Thanks

Joe
David Christensen
2024-08-31 22:30:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
When I input that string into a computer running Debian, it produces an
error message:

2024-08-31 13:07:57 ***@laalaa ~
$ echo $PS1 ; cat /etc/debian_version ; uname -a
\n\D{%Y-%m-%d %H:%M:%S} ${USER}@\h \w\n\$
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10)
x86_64 GNU/Linux

2024-08-31 13:17:14 ***@laalaa ~
$ debian ... amd64-netinst.iso
bash: debian: command not found
Post by John Conover
Can it be coverted to a USB. How?
A Debian Installer "isohybrid" file is compatible with optical media
(e.g. CD-R, DVD-R, or BD-R) and with USB flash drives.


If you wish to create Debian installation media, start by downloading a
Debian Installer isohybrid file:

https://www.debian.org/distrib/


Different computer architectures require different Debian Installer
isohybrid files. Here is the one for 64-bit "Windows PC's":

https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.6.0-amd64-netinst.iso


Burning the Debian Installer isohybrid file onto an optical disc was
easy when most PC's came with Windows, CD-RW drives, and burning
software (e.g. Nero). Today, Rufus seems to be a popular Windows
application for copying ISO images to USB flash drives (untested):

https://www.lifewire.com/how-to-burn-an-iso-file-to-a-usb-drive-2619270


If you have a friend with Linux or BSD, they can use the dd(1) command
to copy the Debian Installer isohybrid file to a USB flash drive:

https://wiki.debian.org/DebianInstaller/CreateUSBMedia


Alternatively, you can purchase Debian Installer media from vendors:

https://www.debian.org/CD/vendors/


Be very careful if you boot the Debian Installer on a computer that
already has another operating system installed. Using the Debian
Installer Rescue Shell can be useful for troubleshooting, but requires
advanced knowledge and skills. Creating and maintaining a "dual boot"
system requires even more knowledge and skills.


I suggest obtaining a used 64-bit "Windows PC" and using it only for Debian.


You can skip the installation media entirely by installing a hypervisor
on your computer, creating a virtual machine (VM), loading the Debian
Installer isohybrid file into the virtual optical disk drive, booting
the VM, and installing Debian. I like VirtualBox:

https://www.virtualbox.org/


I recommend that you buy and read "Learning the Unix Operating System"
by Peek, et al:

https://www.amazon.com/_/dp/0596002610


Then "UNIX and Linux System Administration Handbook" by Nemeth, et al:

https://www.amazon.com/dp/0134277554


David
David Wright
2024-09-01 03:10:02 UTC
Permalink
Post by David Christensen
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
When I input that string into a computer running Debian, it produces
$ echo $PS1 ; cat /etc/debian_version ; uname -a
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10)
x86_64 GNU/Linux
$ debian ... amd64-netinst.iso
bash: debian: command not found
The string "..." is an ellipsis. This one stands for the version number
in any particular ISO file's name.

You can use "…" for an ellipsis in unicode, but it's tedious to insert
it unless you have a Compose sequence set up for it. I suspect most
people don't.

I define it as 'Compose..' myself, and use it a lot to save space.

Cheers,
David.
Timothy M Butterworth
2024-09-01 06:30:01 UTC
Permalink
On Sat, Aug 31, 2024 at 11:15 PM David Christensen <
Post by David Christensen
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
When I input that string into a computer running Debian, it produces an
$ echo $PS1 ; cat /etc/debian_version ; uname -a
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10)
x86_64 GNU/Linux
$ debian ... amd64-netinst.iso
bash: debian: command not found
The iso file is not a command that you run on a Debian system. If you have
cd'd to the present working directory that the iso is located in you still
have to use "./" ./ tells the OS that the file is located in the present
working directory pwd.
Post by David Christensen
Post by John Conover
Can it be coverted to a USB. How?
A Debian Installer "isohybrid" file is compatible with optical media
(e.g. CD-R, DVD-R, or BD-R) and with USB flash drives.
If you wish to create Debian installation media, start by downloading a
https://www.debian.org/distrib/
Different computer architectures require different Debian Installer
https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.6.0-amd64-netinst.iso
Burning the Debian Installer isohybrid file onto an optical disc was
easy when most PC's came with Windows, CD-RW drives, and burning
software (e.g. Nero). Today, Rufus seems to be a popular Windows
https://www.lifewire.com/how-to-burn-an-iso-file-to-a-usb-drive-2619270
If you have a friend with Linux or BSD, they can use the dd(1) command
https://wiki.debian.org/DebianInstaller/CreateUSBMedia
https://www.debian.org/CD/vendors/
Be very careful if you boot the Debian Installer on a computer that
already has another operating system installed. Using the Debian
Installer Rescue Shell can be useful for troubleshooting, but requires
advanced knowledge and skills. Creating and maintaining a "dual boot"
system requires even more knowledge and skills.
I suggest obtaining a used 64-bit "Windows PC" and using it only for Debian.
You can skip the installation media entirely by installing a hypervisor
on your computer, creating a virtual machine (VM), loading the Debian
Installer isohybrid file into the virtual optical disk drive, booting
https://www.virtualbox.org/
I recommend that you buy and read "Learning the Unix Operating System"
https://www.amazon.com/_/dp/0596002610
https://www.amazon.com/dp/0134277554
David
--
⢀⣎⠟⠻⢶⣊⠀
⣟⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄⠀⠀
David Christensen
2024-09-01 08:30:01 UTC
Permalink
Post by Timothy M Butterworth
Post by David Christensen
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
When I input that string into a computer running Debian, it produces an
$ echo $PS1 ; cat /etc/debian_version ; uname -a
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10)
x86_64 GNU/Linux
$ debian ... amd64-netinst.iso
bash: debian: command not found
The iso file is not a command that you run on a Debian system.
Yes -- but the command in the given string is "debian", which is not a
valid command.
Post by Timothy M Butterworth
If you have
cd'd to the present working directory that the iso is located in you still
have to use "./" ./ tells the OS that the file is located in the present
working directory pwd.
"./" is required when a command is in the current working directory and
the current working directory is not in the PATH environment variable.
"./" is not required for file arguments in the current working directory:

2024-09-01 01:11:28 ***@laalaa ~
$ echo 'hello, world!' > hello.txt

2024-09-01 01:11:37 ***@laalaa ~
$ echo 'cat $@' > foo

2024-09-01 01:11:43 ***@laalaa ~
$ chmod +x foo

2024-09-01 01:11:48 ***@laalaa ~
$ ./foo hello.txt
hello, world!


David
Greg Wooledge
2024-09-01 13:20:01 UTC
Permalink
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Yes -- but the command in the given string is "debian", which is not a valid
command.
Stop being so blindly literal. It's pretty obvious to *most* of us
that John was referring to an .iso file, presumably one that he had
downloaded, and now was asking what to do with.

So many of the replies in this thread have been disappointing.

The best reply would be to point John to the Debian Installation Guide:
<https://www.debian.org/releases/bookworm/amd64/>

Specifically, Chapter 4, "Obtaining System Installation Media":
<https://www.debian.org/releases/bookworm/amd64/ch04.en.html>

This tells you what to do with the .iso file, if you're already on a
Linux system with appropriate hardware. It also links to the Debian CD
FAQ, which contains hints on what to do on non-Linux systems.
David Christensen
2024-09-01 21:10:01 UTC
Permalink
Post by Greg Wooledge
Post by John Conover
What does a "debian ... amd64-netinst.iso" do
with an .iso?
Yes -- but the command in the given string is "debian", which is not a valid
command.
Stop being so blindly literal. It's pretty obvious to *most* of us
that John was referring to an .iso file, presumably one that he had
downloaded, and now was asking what to do with.
I agree that the question is broken. Answering it literally points out
that fact and sets the stage for the rest of my post.
Post by Greg Wooledge
So many of the replies in this thread have been disappointing.
There is a saying I heard while working in the electrical construction
trade:

"Doing things right is a matter of experience. Experience is a matter
of doing things wrong."
-- Unknown
Post by Greg Wooledge
<https://www.debian.org/releases/bookworm/amd64/>
<https://www.debian.org/releases/bookworm/amd64/ch04.en.html>
This tells you what to do with the .iso file, if you're already on a
Linux system with appropriate hardware. It also links to the Debian CD
FAQ, which contains hints on what to do on non-Linux systems.
I reviewed chapter 4, section 4.3, and subsection 4.3.1 of the "Debian
GNU/Linux Installation Guide", and estimated that the OP needed simpler
and more concise guidance. So, I wrote the rest of my post.


David
Thomas Schmitt
2024-09-04 13:40:01 UTC
Permalink
Hi,
"shellcheck -e SC2006" (to silence complains concerning ``) suggests double
quotes around "$file"
sum_from_file=`dd if=$file bs=2048 count=$blocks | $checksummer | head -1 |
awk '{print $1}'`
Indeed. Some variable evaluations slipped through unquoted.
Two others are intentionally capable of becomming zero to many words and
as safe as can be without excluding some old shells:
grep $update_v
| $checksummer |

I dimly remember that $(...) did not work on one of the tested machines
back in 2011. I don't see hard reasons to avoid it in the particular
use cases in check_debian_iso. So i keep it as is and hope that the old
tests are not invalidated by the other changes.

shellcheck on Debian 12 also complains about SC2197 (fgrep) and
SC2003 (expr), which are normal for an old script. They are not really
wrong. So i keep them for the same reason as the backticks and the
unquoted variables.


I prepared a new version and put it into libisoburn's git:

https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso-dd-target/check_debian_iso

PGP signature file:

https://dev.lovelyhq.com/libburnia/libisoburn/raw/branch/master/xorriso-dd-target/check_debian_iso.asc

------------------------------------------------------------------------

--- a/check_debian_iso 2011-12-17 19:33:50.000000000 +0100
+++ b/check_debian_iso 2024-09-04 14:19:43.987633803 +0200
@@ -1,6 +1,6 @@
#!/bin/sh
#
-# check_debian_iso, copyright 2011 Thomas Schmitt <***@gmx.net>
+# check_debian_iso, copyright 2011,2024 Thomas Schmitt <***@gmx.net>
# License: GPLv2 or later
# Tested on: Little-endian GNU/Linux with bash
# Little-endian FreeBSD-8 with sh and "md5 -q"
@@ -145,12 +145,12 @@ fi

# The two 16 bit numbers, which are of the appropriate byte sex,
# get combined to a 32 bit number.
-blocks=`expr $lo + $hi '*' 65536`
+blocks=`expr "$lo" + "$hi" '*' 65536`

echo "Piping $blocks blocks of '$file' through '$checksummer'" >&2
echo "to verify checksum list item '$name_from_list'." >&2

-sum_from_file=`dd if=$file bs=2048 count=$blocks | $checksummer | head -1 | awk '{print $1}'`
+sum_from_file=`dd if="$file" bs=2048 count="$blocks" | $checksummer | head -1 | awk '{print $1}'`

if test "$sum_from_list" = "$sum_from_file"
then

------------------------------------------------------------------------

Regrettably the version in
https://people.debian.org/~danchev/debian-iso/
is out of my reach. I will have to ask debian-cd to adopt the new one.
(For now it is only the user who can shoot the own foot by submitting
a hopeless image name.)
Should the command line be prefixed with "#" instead since regular user can
not do it?
Indeed. I had a script enable rw before copying the test ISO to the USB
stick. So i did not notice that superuser authority is needed.
On the other hand
$ sudo mount "$path_to_image_or_usb_device" "$mountpoint"
can be done as a regular user (an alternative is pmount)
udisksctl mount -b /dev/sdb1
I really dislike udisks.
"d-live 12.7.0 kd amd64" is a bit weird from my point of view for the
mountpoint, but unfortunately udisksd logic is hardcoded.
That's one of my reasons.

In general i think that "sudo" makes clearer what obstacle needs to be
surpassed.

Thanks for the review.


Have a nice day :)

Thomas
Franco Martelli
2024-09-04 15:20:01 UTC
Permalink
Hi,
due to popular resistence i created a new wiki page
https://wiki.debian.org/VerifyISOImage
with the content which i would propose to
https://www.debian.org/CD/faq/#verify
Regrettably i was unable to mimick the bullet list paragraphs of the FAQ,
so that i had to change the text for a more unstructured presentation.
(This does not have to be a disadvantage. But now i have two versions
which i need to maintain until a decision is made at debian-cd.)
Interested people are invited to proof-read it, but please make only
cosmetic changes and discuss any substantial change with me in advance.
Technical correctness and portability beyond the currently supported
releases of Debian are important to me.
First of all: thanks Thomas, this is a useful page for newcomers.
I was testing the procedure in a virtual machine, the first "gpg
--keyserver keyring.debian.org ..." command fails because I hadn't
imported the public keys.

If you agree, a paragraph like the following it could be useful at the
beginning:

--- --- --- ---

Before you begin, if you haven't already done, you must import the
Debian public keys. You can download the keys from the authenticity
verification page: https://www.debian.org/CD/verify once done, then
imports the keys with the command:

gpg --import key-988021A964E6EA7D.txt key-DA87E80D6294BE9B.txt
key-42468F4009EA8AC3.txt

--- --- --- ---

Just my 2¢ hint, cheers
--
Franco Martelli
Thomas Schmitt
2024-09-04 17:20:01 UTC
Permalink
Hi,
I was testing the procedure in a virtual machine, the first "gpg --keyserver
keyring.debian.org ..." command fails because I hadn't imported the public
keys.
Hm. I expected that
--keyserver keyring.debian.org
would avoid the need for importing keys to the local keyring.
But the man page indeed does not promise this.

I ran
gpg --list-keys
and really, the key DF9B9C49EAA9298432589D76DA87E80D6294BE9B is listed
as being in
$HOME/.gnupg/pubring.kbx

Moving that file away causes failure to verify.

Thanks for reviewing and testing.
Before you begin, if you haven't already done, you must import the Debian
public keys. You can download the keys from the authenticity verification
page: https://www.debian.org/CD/verify once done, then imports the keys with
gpg --import key-988021A964E6EA7D.txt key-DA87E80D6294BE9B.txt
key-42468F4009EA8AC3.txt
Probably i once followed my own example at
https://www.gnu.org/software/xorriso/#download

Replaying it with end pieces of the Debian keys:

gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3

fixes the verification for me.

So i changed the wiki page to

gpg --keyserver keyring.debian.org --recv-keys 64E6EA7D 6294BE9B 09EA8AC3
gpg --with-fingerprint --verify SHA512SUMS.sign SHA512SUMS

and will do with the debian-cd proposal. (Damn. I hit "Save Changes"
instead of "Preview". I must slow down. Rush brings bad luck.)


Actually i do not know how option --with-fingerprint came to me.
On Debian 12, verification works the same with and without it.
So it could be cargo cult. But most such cult has true roots in the past.
The man page is fewly enlightening with that option.

Does anybody know what benefit it is/was supposed to bring ?
If it is obsolete, since when ?


Have a nice day :)

Thomas
Loading...