why reliable linux hasn't gained more market share?

[David]

crowdstrike makes news headlines, many Windows become blue screens
it is evident that many people around still use Windows
i wonder if linux is more reliable than Windows
according to some statistics linux has only 4% desktop market, 73%
for
MS, 15% for MacOS

Market share is not a reliable recommendation for quality.
How much market share do Rolls Royce or Bugatti have?

why free OS hasn't gained more share even after 30 years of
development?

Because people don't have it hammered into them via the educational
formats, it doesn't come preinstalled on almost every computer you buy:
offered as the only option, Linux isn't advertised, and probably never
will be.
Basically, all the same reasons that Mac is the only option offered in
almost every design school.
Cheers!

[t***@tuxteam.de]

[...]

Post by David

why free OS hasn't gained more share even after 30 years of
development?

Because people don't have it hammered into them via the educational
offered as the only option, Linux isn't advertised, and probably never
will be.

All of them good factors. I may add yet another: because in the current
economic ideology, investing in things seems preferrable than investing
in people -- and Windows (and MacOS) were marketed as "can be administered
by anyone". Which, of course, as often in marketing, is a lie.

Cheers

[Michael Grant]

My opinions only...

1) MS Office (Word/Excel/PPT/etc) has never been available for
Unix/Gnu-Linux. Word and Excel have long been 2 apps users require.
Not OpenOffice. While OpenOffice is quite featureful, it is not 100%
bug for bug compatible with real MS Office products. Similar for
Outlook vs say Thunderbird with respect to the way Outlook is integrated
into the MS universe.

2) Windows vs Unix/Gnu-Linux, Windows is a single operating system.
Whereas on the Unix/Gnu-Linux side you have so many choices it's
overwhelming. Different distros, you have several pure Unix variants,
multiple Linux variants for the underlying OS and then you have
X-Windows with it's myriad of choices. There is no clear single choice.
And then there's the different packaging systems...

3) X-Windows, though as cool as it is to be able to run things remotely
and display them locally, this is rarely used--most individual users
will never use that functionality. Aside from that, X-windows is an
unmitigated disaster from a UX perspective. X's original underlying
programming interface left it up to the programmer to do everything.
This caused every early programs to look and work differently without
any consistency. To fix this, toolkits came along and along with the
toolkits came the toolikit wars and then the window manager wars and
then the wars between Gnome and KDE and other desktops (desktop wars?).
Even multiple ways copypaste works. From a user point of view nothing
is consistent across all apps on Unix/Gnu-Linux and X-Windows. All of
this has kept Unix/Gnu-Linux and X in the "geek space".

4) I've not see a single X-windows based desktop that looked as slick
and as polished as modern Windows or MacOS. Everything seems to just
look and work more clunkily and a bit slower. This is very much my
aesthetic opinion, I know. Things like consistent font sizes and icons
and their proportion and slickness. All very subjective I realize but
in my opinion, this too has made the difference. The "wow" factor just
isn't there. There isn't even a single approximate "look and feel" to a
graphical UI on top of all Unix/Gnu-Linux systems that one could point
to, though some are more popular than others.

There have been efforts to standardize things in the Unix space like
Posix and The Open Group but again, without a single consistent user
paradigm. The people in this space have rallied around choice and not
trying to get programmers to write to one standard but let programmers
create. I have sat on Posix committees and the standards that got
written were to include everything rather than narrow it down to the
best thing to do. Many people have told me over the years that they
really appreciate the diversity of the way applications work under X
windows, that each one has a different UX, some with scroll bars on
left, some on right by default, some square buttons, some rounded,
nothing the same from one to the next. This "wild wild west" approach
has kept Unix/Gnu-Linux from being more mainstream.

5) There is less main stream software available for Unix/Gnu-Linux. As
mentioned above the MS tools suite. Most of the Adobe tools like
Photoshop. Financial tools like Quicken. Some of these things have
moved to online web-based tools. Web based MS Office tools are
definitely not the same as the real ones though. You can argue that
there's a replacement for almost every tool like Gimp for Photoshop but
it's not Photoshop. Most photographers have heard of or used Photoshop,
but not many know or know about Gimp. These are just a few examples,
there are many others. This effect has a knock-on effect of lower
uptake for Unix/Gnu-Linux.

6) Support. Who does the non-technical user go to for tech support?

Since the Unix/Gnu-Linux OS and windowing tools were developed all over
the place, not in some walled garden of Microsoft or Apple, this is why
all this competing and inconsistency has occurred. It's great that we
have Unix/Gnu-Linux don't get me wrong. I'm just giving you my opinion
of the history of why a single Unix or Gnu-Linux system has never had
the same uptake as Windows or MacOS has.

So some mainstream things ARE Unix/Gnu-Linux... MacOS is Unix based, or
at least Mach which has it's lineage from Unix, so there's a mainstream
Unix based OS. But you can't just run MacOS things on anything other
than MacOS (not easily anyway). Android is Linux based and you can get
Android "chrome books". There is Ubuntu and a few other packaged Linux
based OSes (Ubuntu mostly but probably also RedHat) that sometimes ship
on computers but they're never nearly as popular as Windows. Why?
Mostly see (1) above in my opinion. And also you have sheer momentum
behind Windows and MacOS which is hard to get traction foothold in.
Unix/Gnu-Linux (mostly Gnu-Linux as far as I'm aware) is used behind the
scene of many many hardware devices.

7) Once most people buy a computer and it's shipped with an OS, not very
many will wipe it out and install a different OS. MS knows this and
they get hardware vendors to ship Windows.

I think Unix/Gnu-Linux with all it's diversity and openness is great!
Without some unifying force, I just don't see an easy way a fully free
and open system is going to become a mainstream OS used on
desktops/laptops, though Google has managed to do this for phones,
tablets, and some "chrome books", so maybe that's the future, who
knows.....

These are my opinions of why we haven't historically see Unix or
Gnu-Linux running on more computers sitting on mainstream
laptops/desktops. I'm sure some people will disagree with me and will
correct me if I've gotten some of my facts wrong above or forgotten
something important, so feel free to add/correct.

Michael Grant

[George at Clug]

Well said, Michael.

Post by Michael Grant
My opinions only...
1) MS Office (Word/Excel/PPT/etc) has never been available for
Unix/Gnu-Linux. Word and Excel have long been 2 apps users require.
Not OpenOffice. While OpenOffice is quite featureful, it is not 100%
bug for bug compatible with real MS Office products. Similar for
Outlook vs say Thunderbird with respect to the way Outlook is integrated
into the MS universe.
2) Windows vs Unix/Gnu-Linux, Windows is a single operating system.
Whereas on the Unix/Gnu-Linux side you have so many choices it's
overwhelming. Different distros, you have several pure Unix variants,
multiple Linux variants for the underlying OS and then you have
X-Windows with it's myriad of choices. There is no clear single choice.
And then there's the different packaging systems...
3) X-Windows, though as cool as it is to be able to run things remotely
and display them locally, this is rarely used--most individual users
will never use that functionality. Aside from that, X-windows is an
unmitigated disaster from a UX perspective. X's original underlying
programming interface left it up to the programmer to do everything.
This caused every early programs to look and work differently without
any consistency. To fix this, toolkits came along and along with the
toolkits came the toolikit wars and then the window manager wars and
then the wars between Gnome and KDE and other desktops (desktop wars?).
Even multiple ways copypaste works. From a user point of view nothing
is consistent across all apps on Unix/Gnu-Linux and X-Windows. All of
this has kept Unix/Gnu-Linux and X in the "geek space".
4) I've not see a single X-windows based desktop that looked as slick
and as polished as modern Windows or MacOS. Everything seems to just
look and work more clunkily and a bit slower. This is very much my
aesthetic opinion, I know. Things like consistent font sizes and icons
and their proportion and slickness. All very subjective I realize but
in my opinion, this too has made the difference. The "wow" factor just
isn't there. There isn't even a single approximate "look and feel" to a
graphical UI on top of all Unix/Gnu-Linux systems that one could point
to, though some are more popular than others.
There have been efforts to standardize things in the Unix space like
Posix and The Open Group but again, without a single consistent user
paradigm. The people in this space have rallied around choice and not
trying to get programmers to write to one standard but let programmers
create. I have sat on Posix committees and the standards that got
written were to include everything rather than narrow it down to the
best thing to do. Many people have told me over the years that they
really appreciate the diversity of the way applications work under X
windows, that each one has a different UX, some with scroll bars on
left, some on right by default, some square buttons, some rounded,
nothing the same from one to the next. This "wild wild west" approach
has kept Unix/Gnu-Linux from being more mainstream.
5) There is less main stream software available for Unix/Gnu-Linux. As
mentioned above the MS tools suite. Most of the Adobe tools like
Photoshop. Financial tools like Quicken. Some of these things have
moved to online web-based tools. Web based MS Office tools are
definitely not the same as the real ones though. You can argue that
there's a replacement for almost every tool like Gimp for Photoshop but
it's not Photoshop. Most photographers have heard of or used Photoshop,
but not many know or know about Gimp. These are just a few examples,
there are many others. This effect has a knock-on effect of lower
uptake for Unix/Gnu-Linux.
6) Support. Who does the non-technical user go to for tech support?
Since the Unix/Gnu-Linux OS and windowing tools were developed all over
the place, not in some walled garden of Microsoft or Apple, this is why
all this competing and inconsistency has occurred. It's great that we
have Unix/Gnu-Linux don't get me wrong. I'm just giving you my opinion
of the history of why a single Unix or Gnu-Linux system has never had
the same uptake as Windows or MacOS has.
So some mainstream things ARE Unix/Gnu-Linux... MacOS is Unix based, or
at least Mach which has it's lineage from Unix, so there's a mainstream
Unix based OS. But you can't just run MacOS things on anything other
than MacOS (not easily anyway). Android is Linux based and you can get
Android "chrome books". There is Ubuntu and a few other packaged Linux
based OSes (Ubuntu mostly but probably also RedHat) that sometimes ship
on computers but they're never nearly as popular as Windows. Why?
Mostly see (1) above in my opinion. And also you have sheer momentum
behind Windows and MacOS which is hard to get traction foothold in.
Unix/Gnu-Linux (mostly Gnu-Linux as far as I'm aware) is used behind the
scene of many many hardware devices.
7) Once most people buy a computer and it's shipped with an OS, not very
many will wipe it out and install a different OS. MS knows this and
they get hardware vendors to ship Windows.
I think Unix/Gnu-Linux with all it's diversity and openness is great!
Without some unifying force, I just don't see an easy way a fully free
and open system is going to become a mainstream OS used on
desktops/laptops, though Google has managed to do this for phones,
tablets, and some "chrome books", so maybe that's the future, who
knows.....
These are my opinions of why we haven't historically see Unix or
Gnu-Linux running on more computers sitting on mainstream
laptops/desktops. I'm sure some people will disagree with me and will
correct me if I've gotten some of my facts wrong above or forgotten
something important, so feel free to add/correct.
Michael Grant

[Michel Verdier]

OpenOffice is quite featureful, it is not 100% bug for bug compatible with
real MS Office products.

I failed to read an old version word file on a newer word. And succeed
with libreoffice. So yes it's not 100% bug compatible :)

choices. There is no clear single choice. And then there's the different
packaging systems...

Differences and choices are a good thing for evolution

4) I've not see a single X-windows based desktop that looked as slick and as
polished as modern Windows or MacOS. Everything seems to just look and work
more clunkily and a bit slower.

You don't search the right place. Better than windows and macos exists and
works much better.

6) Support. Who does the non-technical user go to for tech support?

I never found a *free* windows support. I got much for debian :)

But I stop here for this so obvious disinformation troll. Same thing for
George at Clug.

[Nicholas Geovanis]

Post by t***@tuxteam.de
[...]

Post by David

why free OS hasn't gained more share even after 30 years of
development?

Because people don't have it hammered into them via the educational
offered as the only option, Linux isn't advertised, and probably never
will be.

Both writers are ignoring the places where the vast majority of Linux
images run:
The corporate data center.
Linux rules the corporate data center and cloud these days. Not so much
Debian there but plenty of Ubuntu and Red Hat/fedora/CentOS.

All of them good factors. I may add yet another: because in the current

Post by t***@tuxteam.de
economic ideology, investing in things seems preferrable than investing
in people --

Any "capital good" like a semi-tractor or a corporate server and the
software on it is "depreciated": We pretend that it lost 6% or more of its
value each year, and we let the corporation write that "loss" off its taxes.

But I'm not allowed to do the same with my car or with the Dell Poweredge
R710 sitting next to me that used to live in the world's largest data
center.

This isn't really ideology except where ideology permits tax cheats to
thrive. Capitalism does that for tax cheats who have power and wealth, not
so much for those who dont.

and Windows (and MacOS) were marketed as "can be administered

Post by t***@tuxteam.de
by anyone". Which, of course, as often in marketing, is a lie.
Cheers
--
t

[Michel Verdier]

i wonder if linux is more reliable than Windows

no doubt :)

according to some statistics linux has only 4% desktop market, 73% for MS, 15%
for MacOS

Linux is not on the market. I buy M$ but download debian. How can you say
how many people is using debian? Once upon a time there was a
linuxcounter...

[Michael Kjörling]

statistics about market share might come from web servers and game servers,
they know how many users use linux and Windows.

No. They at most can know what platform user agents report.

Which isn't necessarily the same thing at all.

[Michel Verdier]

Post by Michael Kjörling

statistics about market share might come from web servers and game servers,
they know how many users use linux and Windows.

No. They at most can know what platform user agents report.

There is also some web server surveys with some stats. For exemple
https://www.netcraft.com/blog/june-2024-web-server-survey/
where you see that apache and nginx are clearly leaders.

I also read bind reaches 60% (80% ?) of dns servers, but I failed to
retrieve my source.

[Jeff Pang]

My reason to keep windows is that I can’t play Starcraft under Linux.

[Larry Martell]

I’ve never owned a machine running windows in my life.

[gene heskett]

I’ve never owned a machine running windows in my life.

I've owned one. I needed a lappy I could use with a gps for roadmap, had
the then new XP on it, cleared the disk a week later and put mandrake on
it because XP had no drivers that could run the broadcom radio in it,
should have been a free module update from hp. I don't think that 20
years later there has ever been a driver for that particular radio that
Just Works. The lappy has long since suicided. Typical hp chinese
sourced stuff even before they sold it all to lenovo.

Now there are around 10 linux installs here, half running armbian, they
get better uptimes than x86-64's.

Cheers, Gene Heskett, CET.

[Jeff Pang]

I would think linux is better as server OS due to reasons of security,
performance and
Operability etc.

Once aol mail was running on windows. But now aol is merged into yahoo
mail which was originally run on freebsd but now linux mostly.

And the initial hotmail was running on freebsd too IIRC. Thought MS
bought it and changed its running environment to windows.

Google FB and many other huge players are using linux as server OS.

I’ve never owned a machine running windows in my life.

[Nicolas George]

Thank David! market share is important though it isn't "reliable
recommendation for quality": more users attract more programmers, who
develop more apps,

The programmers who are attracted by market share are not necessarily
the ones who are interested in developing quality and/or innovative
software, though.

If they were, you'd have support for software-defined radio signal
processing in FFmpeg, for example.

Regards,

[Michael Kjörling]

Post by Nicolas George

Thank David! market share is important though it isn't "reliable
recommendation for quality": more users attract more programmers, who
develop more apps,

The programmers who are attracted by market share are not necessarily
the ones who are interested in developing quality and/or innovative
software, though.

A lot of paid-for programmer time isn't necessarily for what the
individual programmer _wants_ to do. If one's employer dictates that
their products should support Mac OS and Windows, for example, then
there's usually little that a programmer, no matter how motivated, can
do to extend that support to include Linux; especially if the product
in question is heavily dependent on OS-specific APIs.

And let's not forget how many regularly conflate "common" with
"popular". That something is _common_ doesn't necessarily mean that it
is _popular_; it can rather be simply the choice of least resistance.
To within experimental error Linux is always going to face resistance
on the individual level because switching to Linux involves
_replacing_ something which one _knows is working_ on the hardware in
question (as well as something one has a sense of _knowing how to
use_), which is always going to be a rather big step. Myself, I often
emphasize that yes, Linux is _different_ from Windows, but it's not
necessarily _harder to use_, especially for typical office-style tasks
and after a brief period of adjustment.

That said, I've seen a lot of chatter in the creative communities on
the Fediverse (writers/authors in particular) about switching from
Windows to Linux because of Microsoft's recent Recall debacle. I think
I've personally seen three or four people say things to the effect of
"that's it, I'm switching to Linux"; and several more saying things to
the effect of "when I can no longer run my current version of Windows
on my computer I'm switching to Linux". With regards to this week's
Crowdstrike mess, most people who _can_ switch from Windows to Linux
aren't in a position of even having that software on their systems, so
for them personally switching won't have any impact either way. With
Microsoft's Recall, the situation is somewhat different.

[jeremy ardley]

Post by Michael Kjörling

Post by Nicolas George

Thank David! market share is important though it isn't "reliable
recommendation for quality": more users attract more programmers, who
develop more apps,

The programmers who are attracted by market share are not necessarily
the ones who are interested in developing quality and/or innovative
software, though.

A lot of paid-for programmer time isn't necessarily for what the
individual programmer_wants_ to do. If one's employer dictates that
their products should support Mac OS and Windows, for example, then
there's usually little that a programmer, no matter how motivated, can
do to extend that support to include Linux; especially if the product
in question is heavily dependent on OS-specific APIs.

There are plenty of applications that run O/S agnostic.

The earliest were the utterly awful apps in Java that thankfully are now
biting the dust - "Write Once Run Anywhere" actually meant Write Once
and run anywhere the identical JVM is in place and the identical O/S.

A while later QT came along and a lot of software uses the QT API fairly
successfully.

Even later Javascript/Typescript have popped up so applications like
Visual Studio Code run seamlessly on different O/S

And of course Python is now the language du jour and runs equally well
on Windows and Linux especially in the AI realm.

[Michael Kjörling]

Post by jeremy ardley

Post by Michael Kjörling
A lot of paid-for programmer time isn't necessarily for what the
individual programmer_wants_ to do. If one's employer dictates that
their products should support Mac OS and Windows, for example, then
there's usually little that a programmer, no matter how motivated, can
do to extend that support to include Linux; especially if the product
in question is heavily dependent on OS-specific APIs.

There are plenty of applications that run O/S agnostic.

Yes. And there are plenty of (quite possibly a far larger number of)
applications which require one of a small set of particular operating
systems, especially once you get into specialized expert tools; and
even people who need those particular applications for their
day-to-day work, and who _can't_ easily switch to an alternative
implementation of the same general concept.

That there exist counterexamples doesn't help those who _need_ to run
applications which don't run well - or at all - under Linux.

And it puts quite a lot of people off to be told "just switch to an
open-source alternative, it's easy" when they mention that their
day-to-day use requires _particular, specific_ applications which are
only available for proprietary operating systems; often without even
naming them or what those applications do, sometimes because they are
so specialized that few outside of some specialized field would even
recognize the name, much less be able to intelligently suggest
alternatives.

Don't get me wrong; I advocate for Free alternatives where those are
reasonable. Most people don't actually need specialized tools, and for
a large subset of those who do, reasonable alternatives _do_ indeed
exist. But quite a few do need specific tools that _aren't_
cross-platform, and failing to recognize that reflects poorly on
_everyone_.

[gene heskett]

Post by Nicolas George

Thank David! market share is important though it isn't "reliable
recommendation for quality": more users attract more programmers, who
develop more apps,

The programmers who are attracted by market share are not necessarily
the ones who are interested in developing quality and/or innovative
software, though.
If they were, you'd have support for software-defined radio signal
processing in FFmpeg, for example.

Which the current rules for such does not allow, by FCC edicts, only
sealed FCC approved blobs are allowed to play in the rf field.
So don't blame the coders, blame the regukatory agencies.

Post by Nicolas George
Regards,

Cheers, Gene Heskett, CET.

[Nicolas George]

Post by Nicolas George
If they were, you'd have support for software-defined radio signal
processing in FFmpeg, for example.

Which the current rules for such does not allow, by FCC edicts, only sealed
FCC approved blobs are allowed to play in the rf field.
So don't blame the coders, blame the regukatory agencies.

signal processing ≠ emitting

[Hans]

Which is not quite correct. As a hamradio (I am one), you are allowed to
develop your very owh rf-devices. Transceivers, measure equipment, whatever
you like.

Many things, we are using today in consumer devices are first developed by
radio amateurs (example shorthand "packet radio", which is data over hf).

When you have a radio amateur license, you can do lots of things in the air.
Sure, there are regulations, you are not allowed to transmit anywhere and your
transmit power is reduced to 750W, but this does not technical restrict you.

Hamradio is the freedom in the air, you have in coding in linux. Also here are
some rules (GPL, ethicness, kindness whatever), but those do not techniocal
restrict you in any way.

Best regards

Hans

Post by gene heskett
Which the current rules for such does not allow, by FCC edicts, only
sealed FCC approved blobs are allowed to play in the rf field.
So don't blame the coders, blame the regukatory agencies.

Post by Nicolas George
Regards,

Cheers, Gene Heskett, CET.

[George at Clug]

crowdstrike makes news headlines, many Windows become blue screens

The CrowdStrike issue was not a Windows issue, it was a CrowdStrike
issue.

The problem did not affect our Windows computers as we have not
installed CrowdStrike software.

I think the media have a habit of over exaggerating things.

I am not long back from shopping at a supermarket, I asked if they
were affected. Well they were, but not for long as their IT staff
worked furiously to apply the CrowdStrike fix, and soon had things
working again. Not sure how long they were out for, but it did not
adversely affect me, in fact I would never had known if not for the
media hype.

At least I was not travelling on any flights at the time the faulty
update had been pushed. I can wait a day to go buy food, but changing
flights while travelling is something you do not want delays with.

it is evident that many people around still use Windows

I would agree that Windows is the most used OS for desktop PCs.

i wonder if linux is more reliable than Windows

Do you think Windows is not reliable?  Why is that?

according to some statistics linux has only 4% desktop market, 73%

for

MS, 15% for MacOS

Windows is loosing ground?, they have over 90% market share once, when
I was checking out stats.

why free OS hasn't gained more share even after 30 years of

development?

Do you use Linux yourself?

If not, why not?

Have you tried to convincing any Windows users into moving to Linux?

The usual reasons I am given from Windows users are:  

1) They see Microsoft Office as a necessity so then can share
documents with other people. Or they want to use Outlook as their
email client.
(a benefit of having market dominance with a product that can only
effectively run on your own OS)

2) Windows Users believe Windows has more real-time virus scanners
than Linux does.  Please remind me of the list of real-time virus
scanners available for Linux.

3) One thing that concerns me when I try to recommend Linux to Windows
users, is that I cannot get by without using terminal commands in
Linux, but in Windows powershell and command prompt are not required
to be used by standard users. Is it possible to use Linux only from
GUI programs? Many Windows users I know struggle just finding where
their photos are.

4) Software which runs on Windows but is not available on Linux. Photo
shop, various games, etc.   (I am curious how Windows on Copilot+ PC
will go, I expect companies will eventually recompile their software
for the new Snapdragon hardware, but unlikely to rewrite their
software for Linux)

For me, Linux has and does all I require, and I don't mind using
terminal commands now and then. But I am unable to recommend Linux to
anyone who does not 'want' to use it.

[jeremy ardley]

crowdstrike makes news headlines, many Windows become blue screens

The CrowdStrike issue was not a Windows issue, it was a CrowdStrike issue.
The problem did not affect our Windows computers as we have not
installed CrowdStrike software.
I think the media have a habit of over exaggerating things.

The problem was not CrowdStrike as such. It happens in the best of
operations.

The problem is the Windows Systems Administrators who contracted for /
allowed unattended remote updates of kernel drivers on live hardware
systems. This is the height of folly and there is no recovery if it
causes a BSOD.

The situation is recoverable if all the windows machines are virtual
with a good backup/restore plan. The situation is not recoverable if the
kernel updates are on raw iron running Windows.

Heads should roll but obviously won't

[The Wanderer]

Post by jeremy ardley

crowdstrike makes news headlines, many Windows become blue
screens

The CrowdStrike issue was not a Windows issue, it was a CrowdStrike issue.
The problem did not affect our Windows computers as we have not
installed CrowdStrike software.
I think the media have a habit of over exaggerating things.

The problem was not CrowdStrike as such. It happens in the best of
operations.
The problem is the Windows Systems Administrators who contracted for
/ allowed unattended remote updates of kernel drivers on live
hardware systems. This is the height of folly and there is no
recovery if it causes a BSOD.

Speaking as someone who administers (part of) a CrowdStrike Falcon
deployment at my workplace, although I was not involved in selecting it
and would not be able to decide to switch to something else: I do not
believe this is a fair description of what happened.

CrowdStrike Falcon does not manage kernel drivers in general. It manages
its own locally-installed client, which happens to include some
kernel-level drivers. The update in this case does not appear to have
actually modified any of those drivers; it appears to have added a new
data file for use by such a driver, and those data files appear to be
misleadingly named in such a way that they look like drivers.

(I have not confirmed that personally yet, although I have access to the
files in question and intend to do so, but people who are more familiar
with Windows drivers than I am have stated that the files in question do
not comport with the binary file format used by Windows driver files.)

All the sysadmins involved did is agree to let an antivirus-equivalent
utility update itself, and its definitions. I would be surprised if this
could not have easily happened with *any* antivirus-type utility which
has self-update capability; I'm fairly sure all modern broad-spectrum
antivirus-etc. suites on Windows do kernel-level access in similar
fashion. CrowdStrike just happens to be the company involved when it
*did* happen.

That the sysadmins decided to deploy CrowdStrike does not make it
reasonable to fault them for this consequence, any more than e.g. if a
gamer decided to install a game, and then the game required a patch to
let them keep playing, and that patch silently included new/updated DRM
which installed a driver which broke the system (as I recall some past
DRM implementations have reportedly done), it would then be reasonable
to fault the gamer. In neither case was the consequence foreseeable from
the decision.

Post by jeremy ardley
The situation is recoverable if all the windows machines are virtual
with a good backup/restore plan. The situation is not recoverable if
the kernel updates are on raw iron running Windows.

The situation is trivially recoverable if you can get access to the
machine in a way which lets you either boot to safe mode and get
local-administrator access, or lets you boot an alternative environment
(e.g. live-boot media) from which you can read and write to the hard
drive.

I've spent a fair chunk of my workday today going around to affected
computers and performing a variant of the latter process.

Once you've done that, the fix is simple: delete, or move out of the
way, a single file whose name claims that it's a driver. With that file
gone, you can reboot, and Windows will come up normally without the
bluescreen.

Post by jeremy ardley
Heads should roll but obviously won't

What good would decapitation do, here? At most, CrowdStrike's people are
guilty of rolling out an insufficiently-tested update, or of designing a
system such that it's too easy for an update to break things in this
way, or that it's possible to break things in this way not with an
actual new client version (which goes through a release cascade, with
each organization deciding which of the most recent three versions each
of their computers will get) but just with a data-files update (which,
as we have seen here, appears to go out to all clients regardless of
version).

The first would be poor institutional practice; the others would be
potentially-questionable software design, although it's hard to know
without seeing the internal architecture of the software in question and
understanding *why* it's designed that way.

In either case, it's not obvious to me why decapitating a few scapegoats
would *improve* the situation going forward, unless it can be determined
that specific people were actually negligent.

[George at Clug]

Post by The Wanderer

Post by jeremy ardley

crowdstrike makes news headlines, many Windows become blue
screens

The CrowdStrike issue was not a Windows issue, it was a CrowdStrike issue.
The problem did not affect our Windows computers as we have not
installed CrowdStrike software.
I think the media have a habit of over exaggerating things.

The problem was not CrowdStrike as such. It happens in the best of
operations.
The problem is the Windows Systems Administrators who contracted for
/ allowed unattended remote updates of kernel drivers on live
hardware systems. This is the height of folly and there is no
recovery if it causes a BSOD.

Speaking as someone who administers (part of) a CrowdStrike Falcon
deployment at my workplace, although I was not involved in selecting it
and would not be able to decide to switch to something else: I do not
believe this is a fair description of what happened.
CrowdStrike Falcon does not manage kernel drivers in general. It manages
its own locally-installed client, which happens to include some
kernel-level drivers. The update in this case does not appear to have
actually modified any of those drivers; it appears to have added a new
data file for use by such a driver, and those data files appear to be
misleadingly named in such a way that they look like drivers.
(I have not confirmed that personally yet, although I have access to the
files in question and intend to do so, but people who are more familiar
with Windows drivers than I am have stated that the files in question do
not comport with the binary file format used by Windows driver files.)
All the sysadmins involved did is agree to let an antivirus-equivalent
utility update itself, and its definitions. I would be surprised if this
could not have easily happened with *any* antivirus-type utility which
has self-update capability; I'm fairly sure all modern broad-spectrum
antivirus-etc. suites on Windows do kernel-level access in similar
fashion. CrowdStrike just happens to be the company involved when it
*did* happen.
That the sysadmins decided to deploy CrowdStrike does not make it
reasonable to fault them for this consequence, any more than e.g. if a
gamer decided to install a game, and then the game required a patch to
let them keep playing, and that patch silently included new/updated DRM
which installed a driver which broke the system (as I recall some past
DRM implementations have reportedly done), it would then be reasonable
to fault the gamer. In neither case was the consequence foreseeable from
the decision.

Post by jeremy ardley
The situation is recoverable if all the windows machines are virtual
with a good backup/restore plan. The situation is not recoverable if
the kernel updates are on raw iron running Windows.

The situation is trivially recoverable if you can get access to the
machine in a way which lets you either boot to safe mode and get
local-administrator access, or lets you boot an alternative environment
(e.g. live-boot media) from which you can read and write to the hard
drive.
I've spent a fair chunk of my workday today going around to affected
computers and performing a variant of the latter process.
Once you've done that, the fix is simple: delete, or move out of the
way, a single file whose name claims that it's a driver. With that file
gone, you can reboot, and Windows will come up normally without the
bluescreen.

Post by jeremy ardley
Heads should roll but obviously won't

What good would decapitation do, here? At most, CrowdStrike's people are
guilty of rolling out an insufficiently-tested update, or of designing a
system such that it's too easy for an update to break things in this
way, or that it's possible to break things in this way not with an
actual new client version (which goes through a release cascade, with
each organization deciding which of the most recent three versions each
of their computers will get) but just with a data-files update (which,
as we have seen here, appears to go out to all clients regardless of
version).
The first would be poor institutional practice; the others would be
potentially-questionable software design, although it's hard to know
without seeing the internal architecture of the software in question and
understanding *why* it's designed that way.
In either case, it's not obvious to me why decapitating a few scapegoats
would *improve* the situation going forward, unless it can be determined
that specific people were actually negligent.

Thanks Wanderer,

Please no 'decapitating', or I would have lost my head many years ago, and often (if that is possible).

Testing is important. Like 'backup and restore verification', often considered insufficient in hindsight after an incident, but rarely considered insufficient before the incident.

Even with our best testing, we all make mistakes from time to time, and I have made my fair share.

My aim is not to blame, but it is necessary to identify the cause and to carefully consider how to mitigate further occurrences.

Over reaction is not good - one decision might be not to use anti-virus software, which would mitigate the issue of anti-virus software bugs causing outages, but that would be far worse a solution than an occasional and rare outage.

And as for testing, testing IS necessary, but it will only ever be testing, 1) it is not possible to test for everything, 2) over testing can cause issues too, while still not capturing all potential issues.

I want to thank all the people from CrowdStrike and all the people applying the fix patches, thanks for quickly restoring services. Keep up the great work of protecting our Internet services.

George.

Post by The Wanderer
--
The Wanderer
The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw

[jeremy ardley]

Post by The Wanderer
The first would be poor institutional practice; the others would be
potentially-questionable software design, although it's hard to know
without seeing the internal architecture of the software in question and
understanding*why* it's designed that way.
In either case, it's not obvious to me why decapitating a few scapegoats
would*improve* the situation going forward, unless it can be determined
that specific people were actually negligent.

The CrowdStrike outage emulated the very thing it is alleged to protect
against - a zero day exploit.

The difference is CrowdStrike has a far better distribution mechanism as
all its victims willingly accepted it being put on their machines and
willingly accepted automatic updates, each of which potentially could
cause a failure.

Given the time delays in recovery and in many organisations reports of
people having to drive to physical locations to reset machines there was
clearly no effective mitigation or recovery plans in place.

There are ways to mitigate a zero day exploit such as Out-of-Band
Management (OOBM) or Baseboard Management Controller (BMC) so at least
the system can be recovered, at least remotely, and likely
automatically. Alternatively services can run virtually and can be reset
automatically by monitoring systems.

There is also the system design issues that even if the majority of
systems are immune, key system failures will take down a network. Active
Directory servers seem a particularly weak point.

So my point still stands. Those responsible for mitigation of
faults/zero day exploits in many cases were negligent in their system
and process design. Specifically they did not install hardware and
software that could be remotely and automatically managed out of band
and they provided essential services such as Active Directory on
vulnerable hosts with often no easy way to recover them.

On a second level I do have to ask if CrowdStrike and equivalent
reactive monitoring systems actually provide value? Yes, they reduce the
time a zero day exploit has to be effective, but you have to assume
there *will* be a serious exploit and you *will* lose functionality
and/or data. Focusing on resilience of service, hardening of software,
and management of data that even if stolen is of no value seems to be
more useful.

[Alain D D Williams]

Post by jeremy ardley
The CrowdStrike outage emulated the very thing it is alleged to protect
against - a zero day exploit.

It was also a demonstration of a huge vulnerability. If $EvilActor were to get
an agent employed at CrowdStrike/whoever then they could take down a lot of
important infrastructure world wide. This could give them a great advantage if
done, for instance, just before they invade another country or something.

In a cynical moment on Friday I did wonder if this was a dry run.

Unfortunately I suspect that the lesson will not be learned and that most will
largely forget this before too long.

:-(

[Jeffrey Walton]

Post by The Wanderer

Post by jeremy ardley
[...]
The problem was not CrowdStrike as such. It happens in the best of
operations.
The problem is the Windows Systems Administrators who contracted for
/ allowed unattended remote updates of kernel drivers on live
hardware systems. This is the height of folly and there is no
recovery if it causes a BSOD.

[...]
All the sysadmins involved did is agree to let an antivirus-equivalent
utility update itself, and its definitions. I would be surprised if this
could not have easily happened with *any* antivirus-type utility which
has self-update capability; I'm fairly sure all modern broad-spectrum
antivirus-etc. suites on Windows do kernel-level access in similar
fashion. CrowdStrike just happens to be the company involved when it
*did* happen.

I was around when Symantec Antivirus did about the same to about half
the workstations at the Social Security Administration. A definition
file update blue screened about half the Windows NT 4.0 and Windows
2000 hosts. That was about 50,000 machines, if I recall correctly.

Post by The Wanderer
That the sysadmins decided to deploy CrowdStrike does not make it
reasonable to fault them for this consequence, any more than e.g. if a
gamer decided to install a game, and then the game required a patch to
let them keep playing, and that patch silently included new/updated DRM
which installed a driver which broke the system (as I recall some past
DRM implementations have reportedly done), it would then be reasonable
to fault the gamer. In neither case was the consequence foreseeable from
the decision.

Sysadmins don't make that decision in the Enterprise. That decision
was made above the lowly sysadmin's pay grade.

Post by The Wanderer

Post by jeremy ardley
The situation is recoverable if all the windows machines are virtual
with a good backup/restore plan. The situation is not recoverable if
the kernel updates are on raw iron running Windows.

The situation is trivially recoverable if you can get access to the
machine in a way which lets you either boot to safe mode and get
local-administrator access, or lets you boot an alternative environment
(e.g. live-boot media) from which you can read and write to the hard
drive.

I don't think it's trivial for some enterprises due to the sheer
number of machines and the remote workforce. I'm guessing the company
I work for will spend the next week or month sorting things out. And
the company is a medium size enterprise with about 30,000 employees.
Imagine how bad it's going to be for an enterprise with 100,000
employees.

Post by The Wanderer
I've spent a fair chunk of my workday today going around to affected
computers and performing a variant of the latter process.
Once you've done that, the fix is simple: delete, or move out of the
way, a single file whose name claims that it's a driver. With that file
gone, you can reboot, and Windows will come up normally without the
bluescreen.

Unfortunately, I don't see this as scalable. It works fine for a small
business with 100 employees, but not an enterprise.

Post by The Wanderer

Post by jeremy ardley
Heads should roll but obviously won't

What good would decapitation do, here?

I think it's a figure of speech; not a literal.

Post by The Wanderer
At most, CrowdStrike's people are
guilty of rolling out an insufficiently-tested update, or of designing a
system such that it's too easy for an update to break things in this
way, or that it's possible to break things in this way not with an
actual new client version (which goes through a release cascade, with
each organization deciding which of the most recent three versions each
of their computers will get) but just with a data-files update (which,
as we have seen here, appears to go out to all clients regardless of
version).

At minimum, it is negligence.

Post by The Wanderer
The first would be poor institutional practice; the others would be
potentially-questionable software design, although it's hard to know
without seeing the internal architecture of the software in question and
understanding *why* it's designed that way.
In either case, it's not obvious to me why decapitating a few scapegoats
would *improve* the situation going forward, unless it can be determined
that specific people were actually negligent.

The incident affected the company's share price. Shares were down $10
or $15. If the potential issues were not detailed in company
literature and prospectus, then the Securities and Exchange Commission
might get involved for misrepresenting risk and liabilities. There
could be big fines, and that will cost the shareholders more money.

All this points to an incompetent board. If someone's head is going to
be taken (figuratively), then it should start with the CEO and other
executives.

Jeff

[Bret Busby]

On 21/7/24 10:07, Jeffrey Walton wrote:

<snip>

Post by Jeffrey Walton
All this points to an incompetent board. If someone's head is going to
be taken (figuratively), then it should start with the CEO and other
executives.

Yes.

But, the people who should be sacked, with loss of benefits, are the
board members and the CEO's and the CIO's of the institutions -
government departments and businesses, who were not running Linux or BSD
UNIX instead of MS Windows.

Crowdstrike did not strike at Linux or BSD UNIX systems - only MS
Windows systems.

..
Bret Busby
Armadale
West Australia
(UTC+0800)
..............

[Andy Smith]

Hi,

Crowdstrike did not strike at Linux or BSD UNIX systems - only MS Windows
systems.

Except that time just a few months ago when it *did* happen to
Crowdstrike+Linux?

https://news.ycombinator.com/item?id=41005936

Nothing in this story would be different if it was Linux deployed on
those machines. And nothing would be different if Crowdstrike didn't
exist, as some other equally useless vendor would be involved.

There is a need to examine why companies are putting high privilege
junk software on their machines. It's got nothing to do with Linux
vs Windows.

Thanks,
Andy

[Jeffrey Walton]

Post by Andy Smith

Crowdstrike did not strike at Linux or BSD UNIX systems - only MS Windows
systems.

Except that time just a few months ago when it *did* happen to
Crowdstrike+Linux?
https://news.ycombinator.com/item?id=41005936

This is alarming (to me) from the YC post:

"we push software to your machines any time we want,
whether or not it's urgent, without testing it" seems to be
core to the model...

Updates need to be tested inside an organization's lab, and then
tested with a sampling of the organization's computers. Then, an
organization is free to release the update to all machines. All of
that has to happen in two weeks to 30 days.

Post by Andy Smith
Nothing in this story would be different if it was Linux deployed on
those machines. And nothing would be different if Crowdstrike didn't
exist, as some other equally useless vendor would be involved.
There is a need to examine why companies are putting high privilege
junk software on their machines. It's got nothing to do with Linux
vs Windows.

Jeff

[Richmond]

Post by Jeffrey Walton
"we push software to your machines any time we want,
whether or not it's urgent, without testing it" seems to be
core to the model...
Updates need to be tested inside an organization's lab, and then
tested with a sampling of the organization's computers. Then, an
organization is free to release the update to all machines. All of
that has to happen in two weeks to 30 days.

Yes the updates should be tested at every stage. Maybe people think that
they cannot stop updates, but they can use Group Policy to stop Windows
Update. Or maybe they are afraid if they don't allow virus updates then
they will allow a virus?

[Dan Ritter]

Post by Richmond
Yes the updates should be tested at every stage. Maybe people think that
they cannot stop updates, but they can use Group Policy to stop Windows
Update. Or maybe they are afraid if they don't allow virus updates then
they will allow a virus?

This wasn't Windows Update. This is more akin to Firefox's
Mozilla-owned self-updating.

Are we sufficiently far away from Debian now?

The relevant bits for Debian:

- when you give root privileges to someone, they own your
computer

- software updates that run as root (including Debian updates)
can run anything else as root

- insiders and organizations you hire need to be part of your
security assessment

- intentional and unintentional acts can do the same amount of
damage


-dsr-

-dsr-

[Stefan Monnier]

Post by Dan Ritter
- software updates that run as root (including Debian updates)
can run anything else as root

So, maybe a more relevant discussion is: what will happen when a Debian
stable security update comes with a "big blunder" that crashes the most
machines in early boot?

Admittedly, the wider variety of Debian installs might make the "most"
above much less likely, but it's still something that can
definitely happen.

What does Debian do to try and avoid that, and what do *we* (Debian
users) do to try and mitigate that?


Stefan

[Dan Ritter]

Post by Stefan Monnier

Post by Dan Ritter
- software updates that run as root (including Debian updates)
can run anything else as root

So, maybe a more relevant discussion is: what will happen when a Debian
stable security update comes with a "big blunder" that crashes the most
machines in early boot?
Admittedly, the wider variety of Debian installs might make the "most"
above much less likely, but it's still something that can
definitely happen.
What does Debian do to try and avoid that, and what do *we* (Debian
users) do to try and mitigate that?

Testing is necessary but not sufficient. If you can afford to have a
spare machine or a spare VM that gets upgraded a few days before your
other machines do, and test the heck out of that.

At sufficient scale -- a scale which is within the reach of increasingly
many people as storage costs continue to reduce -- we can keep our own
mirrors of upstream.

-dsr-

[Richmond]

Post by Dan Ritter

Post by Richmond
Yes the updates should be tested at every stage. Maybe people think that
they cannot stop updates, but they can use Group Policy to stop Windows
Update. Or maybe they are afraid if they don't allow virus updates then
they will allow a virus?

This wasn't Windows Update. This is more akin to Firefox's
Mozilla-owned self-updating.

Windows Updates should be tested too. I worded what I said quite
carefully.

[Joe]

On Sun, 21 Jul 2024 02:43:15 -0400

Post by Jeffrey Walton
"we push software to your machines any time we want,
whether or not it's urgent, without testing it" seems to be
core to the model...
Updates need to be tested inside an organization's lab, and then
tested with a sampling of the organization's computers. Then, an
organization is free to release the update to all machines. All of
that has to happen in two weeks to 30 days.

At the very least, updates should be avoided for a few days after
release, unless they are the exceeding rare 'Oh my God, patch this
yesterday' kind, such as when the malware writers of the world realised
that Windows MetaFiles could contain executables...

Small businesses cannot really do as you say, but any business large
enough to have logistics problems in fixing all of their machines
quickly should do so.

[Hans]

I do not agree to this. Updates should be installed as soon as they are
available. Especially security updates. It shows , that within 24 hours after
the release of an update, an exploit is available for this security hole.

But you should do it corrdectly, like some hospitals did: First check with a
canary (a testserver or some unimportant server), then, when everything is
working without any problems, roll it out to the rest of the servers.

Waiting for some days is a very very bad idea!

I admit, that many people do not so, because they are comfortable and this
requires more work. But it is the correct way!

And really: This is not a new knowledge, this practice is standard since years
(or should be everywhere).

If one think, he must not do it and rely on the manufacturer, well his
decision. If it breaks, i have no pity for him.

Best

Hans

Post by Joe
At the very least, updates should be avoided for a few days after
release, unless they are the exceeding rare 'Oh my God, patch this
yesterday' kind, such as when the malware writers of the world realised
that Windows MetaFiles could contain executables...
Small businesses cannot really do as you say, but any business large
enough to have logistics problems in fixing all of their machines
quickly should do so.

[Jeffrey Walton]

Post by Hans
I do not agree to this. Updates should be installed as soon as they are
available. Especially security updates. It shows , that within 24 hours after
the release of an update, an exploit is available for this security hole.

I think you may be conflating two different updates. The first is the
OS or application's updates for a vulnerability, and second is the
antivirus updates to detect an attack using the vulnerability.

The science tells us that most compromised servers happen long after
an exploit is disclosed and patched. The majority of compromises
happen after 90 days, and continue for years afterwards. Confer,
<https://www.cs.umd.edu/~waa/pubs/Windows_of_Vulnerability.pdf>.

So a Patch Management program that tests the OS or application
vendor's updates within about two weeks is usually going to be Ok.
Since it is the OS vendor or application vendor, it might be Ok to be
very aggressive in applying the updates since the OS or application
vendor are the experts for their product. That covers the first case -
OS or application updates for a vulnerability.

The second case is trickier - detecting an attack using the
vulnerability. This is where antivirus comes into play. In my mind's
eye, antivirus companies are an externality/third party, and their
work needs to be tested even more than the OS or application. The
testing needs to be more thorough because the third party does not
have specialized knowledge of the organization or the OS or
application. Yet the third party will likely run with highest of
privileges, and violate a number of the tenets laid out by Saltzer and
Schroeder. Confer,
<https://www.cs.virginia.edu/~evans/cs551/saltzer/>.

Post by Hans
But you should do it corrdectly, like some hospitals did: First check with a
canary (a testserver or some unimportant server), then, when everything is
working without any problems, roll it out to the rest of the servers.

Are the hospitals checking the OS or application updates; or are they
checking the antivirus updates?

Post by Hans
Waiting for some days is a very very bad idea!
I admit, that many people do not so, because they are comfortable and this
requires more work. But it is the correct way!
And really: This is not a new knowledge, this practice is standard since years
(or should be everywhere).
If one think, he must not do it and rely on the manufacturer, well his
decision. If it breaks, i have no pity for him.

Jeff

[Joe]

On Sun, 21 Jul 2024 02:41:35 +0000

Post by Andy Smith
Hi,

Post by Bret Busby
Crowdstrike did not strike at Linux or BSD UNIX systems - only MS
Windows systems.

Except that time just a few months ago when it *did* happen to
Crowdstrike+Linux?
https://news.ycombinator.com/item?id=41005936
Nothing in this story would be different if it was Linux deployed on
those machines. And nothing would be different if Crowdstrike didn't
exist, as some other equally useless vendor would be involved.
There is a need to examine why companies are putting high privilege
junk software on their machines. It's got nothing to do with Linux
vs Windows.

It is in the sense that Linux generally does not need that kind of
software, though Windows need not have done if many design decisions
down the years had been taken differently.

I'd be willing to bet that the market for hugely expensive anti-malware
systems is much smaller for Linux than for Windows, despite the large
number of Linux servers in use. I saw a recent post saying that Linux
had very few anti-virus products, when the reality is that Linux
anti-virus products are targeting *Windows* malware.

[The Wanderer]

Post by Jeffrey Walton

Post by The Wanderer

Post by jeremy ardley
The problem is the Windows Systems Administrators who contracted
for / allowed unattended remote updates of kernel drivers on
live hardware systems. This is the height of folly and there is
no recovery if it causes a BSOD.

All the sysadmins involved did is agree to let an
antivirus-equivalent utility update itself, and its definitions. I
would be surprised if this could not have easily happened with
*any* antivirus-type utility which has self-update capability; I'm
fairly sure all modern broad-spectrum antivirus-etc. suites on
Windows do kernel-level access in similar fashion. CrowdStrike just
happens to be the company involved when it *did* happen.

I was around when Symantec Antivirus did about the same to about
half the workstations at the Social Security Administration. A
definition file update blue screened about half the Windows NT 4.0
and Windows 2000 hosts. That was about 50,000 machines, if I recall
correctly.

There *is* a difference between this incident and that one, in the form
of the *scale* of the issue. But otherwise, yes, I've seen less-severe
breakages of this sort occur in the past as well.

Post by Jeffrey Walton

Post by The Wanderer
That the sysadmins decided to deploy CrowdStrike does not make it
reasonable to fault them for this consequence, any more than e.g.
if a gamer decided to install a game, and then the game required a
patch to let them keep playing, and that patch silently included
new/updated DRM which installed a driver which broke the system (as
I recall some past DRM implementations have reportedly done), it
would then be reasonable to fault the gamer. In neither case was
the consequence foreseeable from the decision.

Sysadmins don't make that decision in the Enterprise. That decision
was made above the lowly sysadmin's pay grade.

It does depend on the enterprise. In my organization, I'm fairly sure
the people who made the decision at least did so with informed input
from the sysadmins, including specifically the people who were
administering the existing antivirus solution (McAfee).

Post by Jeffrey Walton

Post by The Wanderer

Post by jeremy ardley
The situation is recoverable if all the windows machines are
virtual with a good backup/restore plan. The situation is not
recoverable if the kernel updates are on raw iron running
Windows.

The situation is trivially recoverable if you can get access to
the machine in a way which lets you either boot to safe mode and
get local-administrator access, or lets you boot an alternative
environment (e.g. live-boot media) from which you can read and
write to the hard drive.

I don't think it's trivial for some enterprises due to the sheer
number of machines and the remote workforce.

Yeah - after the fact it occurred to me that I hadn't specified that
what this is *not* is *automatable*, which has inevitable consequences
for the difficulty of scaling the solution out.

At most you could provide bootable media which would, when booted to,
fix the issue and reboot. (If you could set things up for that to be
available by PXE boot, and if you have everything configured to try PXE
booting first before booting locally, then maybe you could automate it
with nothing more than telling people to reboot any computer they see
affected? But even that type of solution has its limits.)

Post by Jeffrey Walton
I'm guessing the company I work for will spend the next week or month
sorting things out. And the company is a medium size enterprise with
about 30,000 employees. Imagine how bad it's going to be for an
enterprise with 100,000 employees.

Oh, I can.

Post by Jeffrey Walton

Post by The Wanderer
I've spent a fair chunk of my workday today going around to
affected computers and performing a variant of the latter process.
Once you've done that, the fix is simple: delete, or move out of
the way, a single file whose name claims that it's a driver. With
that file gone, you can reboot, and Windows will come up normally
without the bluescreen.

Unfortunately, I don't see this as scalable. It works fine for a
small business with 100 employees, but not an enterprise.

My own organization has thousands of computers, something like 1000-3000
of which have CrowdStrike Falcon as their antimalware solution. The part
of our IT department which would typically be expected to handle the
client-side remediation of something like this (including making and
keeping appointments with remote workers who were impacted) is a maximum
of 16 people, and I believe we're currently working with two positions
empty.

That said, a *lot* of our CrowdStrike-using computers seem to have not
been affected; as far as I can tell, most of them were *off* for the
entire active-issue period, and so never received the problematic
update. Someone has estimated that only 8% of our total computers are
affected. (I don't know where they got the figure from, but I do know
that "our total computers" includes another 3000-5000 units which use a
different antimalware solution, so that's going to skew the percentage.)

It's still likely to take us weeks, if not months, to get everything
affected by this back into working order.

Post by Jeffrey Walton

Post by The Wanderer

Post by jeremy ardley
Heads should roll but obviously won't

What good would decapitation do, here?

I think it's a figure of speech; not a literal.

Indeed. I was simply extending the metaphor.

Post by Jeffrey Walton

Post by The Wanderer
At most, CrowdStrike's people are guilty of rolling out an
insufficiently-tested update, or of designing a system such that
it's too easy for an update to break things in this way, or that
it's possible to break things in this way not with an actual new
client version (which goes through a release cascade, with each
organization deciding which of the most recent three versions each
of their computers will get) but just with a data-files update
(which, as we have seen here, appears to go out to all clients
regardless of version).

At minimum, it is negligence.

Agreed.

Post by Jeffrey Walton

Post by The Wanderer
The first would be poor institutional practice; the others would
be potentially-questionable software design, although it's hard to
know without seeing the internal architecture of the software in
question and understanding *why* it's designed that way.
In either case, it's not obvious to me why decapitating a few
scapegoats would *improve* the situation going forward, unless it
can be determined that specific people were actually negligent.

The incident affected the company's share price. Shares were down
$10 or $15.

I was watching this over the course of the day, and saw it quoted
starting at "down nearly 15%" before the start of trading, and "down 9%"
after trading had closed for the day. I'm not sure what that reflects in
real-world practice, and I didn't see dollar prices quoted.

Post by Jeffrey Walton
If the potential issues were not detailed in company literature and
prospectus, then the Securities and Exchange Commission might get
involved for misrepresenting risk and liabilities. There could be big
fines, and that will cost the shareholders more money.
All this points to an incompetent board. If someone's head is going
to be taken (figuratively), then it should start with the CEO and
other executives.

I could see an argument for that, although I'm not convinced 100% based
on what I've seen to date. I'd need more information and details, and am
unlikely to get them.

[Hans]

Hello,

well, the thing is: Do we really want to go to more market share?

Let's imagine, Debian becomes market relevant, what will happen? Sure, more
developers get paid, what is very nice. But not all developers will.

Many good developers will not be paid and when the market will rule things,
then many good developers will be pushed away or demoralied. Because it will
become common, that people will no more cherish theire work.

The development of a few people will be cherished, those, who create programs,
the market wants.

I am using linux since more than 30 years and it is impressive, what people
can do, when they can do, what they want and what they like.

And look at the quality, look, what has been created since the beginning. This
was only possible, because no market forced people, to do things the market
wants, not what the developers want.

I think, we all can be happy, that we are not dependent from any market, the
developers, because theire freedom and theire contentement is not been
deminished, and the users, who get very good and high qulitative software to
work with.

And if you really think, the more you spend, the better the software, you can
of course buy software only from the market.

Or, you can donate linux developers and/or distributors of your money.

Personally(!) I think, the second way is better, because I can speak directly
to developers, could (if I would be capable of) fix things myself together
with the developers and maybe can even ask him, to implenent some functions
especially for me.

All things, a market driven software will never offer.

So, I think, we can be happy, that linux (and debian) is not market relevant.
It will lose its freedom, its high quality and the joy of many people.

Sorry, if I did not always find the right expression, I am not native English.

Best regards

Hans

[George at Clug]

Post by Hans
Hello,
well, the thing is: Do we really want to go to more market share?
Let's imagine, Debian becomes market relevant, what will happen? Sure, more
developers get paid, what is very nice. But not all developers will.
Many good developers will not be paid and when the market will rule things,
then many good developers will be pushed away or demoralied. Because it will
become common, that people will no more cherish theire work.
The development of a few people will be cherished, those, who create programs,
the market wants.
I am using linux since more than 30 years and it is impressive, what people
can do, when they can do, what they want and what they like.
And look at the quality, look, what has been created since the beginning. This
was only possible, because no market forced people, to do things the market
wants, not what the developers want.
I think, we all can be happy, that we are not dependent from any market, the
developers, because theire freedom and theire contentement is not been
deminished, and the users, who get very good and high qulitative software to
work with.
And if you really think, the more you spend, the better the software, you can
of course buy software only from the market.
Or, you can donate linux developers and/or distributors of your money.
Personally(!) I think, the second way is better, because I can speak directly
to developers, could (if I would be capable of) fix things myself together
with the developers and maybe can even ask him, to implenent some functions
especially for me.
All things, a market driven software will never offer.
So, I think, we can be happy, that linux (and debian) is not market relevant.
It will lose its freedom, its high quality and the joy of many people.

Hans, I find much wisdom in your above statement regards 'freedom', thank you, George.

Post by Hans
Sorry, if I did not always find the right expression, I am not native English.
Best regards
Hans

[Joe]

On Sat, 20 Jul 2024 15:59:14 +0200

Post by Hans
Hello,
well, the thing is: Do we really want to go to more market share?
Let's imagine, Debian becomes market relevant, what will happen?
Sure, more developers get paid, what is very nice. But not all
developers will.
Many good developers will not be paid and when the market will rule
things, then many good developers will be pushed away or demoralied.
Because it will become common, that people will no more cherish
theire work.
The development of a few people will be cherished, those, who create
programs, the market wants.
I am using linux since more than 30 years and it is impressive, what
people can do, when they can do, what they want and what they like.
And look at the quality, look, what has been created since the
beginning. This was only possible, because no market forced people,
to do things the market wants, not what the developers want.
I think, we all can be happy, that we are not dependent from any
market, the developers, because theire freedom and theire
contentement is not been deminished, and the users, who get very good
and high qulitative software to work with.
And if you really think, the more you spend, the better the software,
you can of course buy software only from the market.
Or, you can donate linux developers and/or distributors of your money.
Personally(!) I think, the second way is better, because I can speak
directly to developers, could (if I would be capable of) fix things
myself together with the developers and maybe can even ask him, to
implenent some functions especially for me.
All things, a market driven software will never offer.
So, I think, we can be happy, that linux (and debian) is not market
relevant. It will lose its freedom, its high quality and the joy of
many people.
Sorry, if I did not always find the right expression, I am not native English.

You missed one: Linux is virtually a virus-free environment, and a
large user base would mean many more people running as root, and it
would become worth the time of malware writers to target Linux. Linux
would become as virus-ridden as Windows.

It would also become a target for data harvesting, from which Debian,
at least, is refreshingly free. I have no doubt that MS makes more
money from user data sales than it does from sales of domestic versions
of Windows.

[Hans]

Post by Joe
You missed one: Linux is virtually a virus-free environment, and a
large user base would mean many more people running as root, and it
would become worth the time of malware writers to target Linux. Linux
would become as virus-ridden as Windows.
It would also become a target for data harvesting, from which Debian,
at least, is refreshingly free. I have no doubt that MS makes more
money from user data sales than it does from sales of domestic versions
of Windows.

I do not agree. This is an argument, i am often get confronted with. The more
linux, the more malware? No, it isn't. See, linux is the most used OS in the
server world. All important companies rely on it. EBay, Google, Amazon, and
even Microsof. Its DNS running Linux. Cloudflare and others, too.

So, these are really interesting targets, where you can really hurt lots of
people. If linux would bre so easy to crack like Windows, the attackers would
do. But it isn't. It is (mostly) secure by design.

There are millions of "viruses" for Windows, but only a handfull of viruses
(or rootkits) for linux.

And think of OpenBSD: Only 2 security holes in more than 15 years. How many
security holes got Windows in th elast 10-15 years? With all their money,
which can buy any super, duper coder look at the result.

No, I see it else. It can be done (OpenBSD is showing it). It is the arrogance
of Microsoft (and many other companies).

It is not the spread of Windows, it is theire bad quality what makes crackers
attack this system. Low fruits, you know?

And there is another thing, that makes linux better: The developers want to
write stable and secure software. It is theire joy and happiness. They do not
mourn, when someone is telling a bug or a security hole. They are happy, to
fix it. Making theire software, theire "baby" better.

In market, the developers MUST do it, for them fixing software is just
annoying and more work (for the same money). That is the differnce.

Note: I do not want to claim, linux developers are the better coders. But they
are coding with theire heart. That makes the difference.

It is not the spreading of software.

Have fun!

Hans

[Joe]

On Sat, 20 Jul 2024 22:13:00 +0200

Post by Hans

Post by Joe
You missed one: Linux is virtually a virus-free environment, and a
large user base would mean many more people running as root, and it
would become worth the time of malware writers to target Linux.
Linux would become as virus-ridden as Windows.
It would also become a target for data harvesting, from which
Debian, at least, is refreshingly free. I have no doubt that MS
makes more money from user data sales than it does from sales of
domestic versions of Windows.

I do not agree. This is an argument, i am often get confronted with.
The more linux, the more malware? No, it isn't. See, linux is the
most used OS in the server world. All important companies rely on it.
EBay, Google, Amazon, and even Microsof. Its DNS running Linux.
Cloudflare and others, too.
So, these are really interesting targets, where you can really hurt
lots of people. If linux would bre so easy to crack like Windows, the
attackers would do. But it isn't. It is (mostly) secure by design.
There are millions of "viruses" for Windows, but only a handfull of
viruses (or rootkits) for linux.
And think of OpenBSD: Only 2 security holes in more than 15 years.
How many security holes got Windows in th elast 10-15 years? With all
their money, which can buy any super, duper coder look at the result.
No, I see it else. It can be done (OpenBSD is showing it). It is the
arrogance of Microsoft (and many other companies).
It is not the spread of Windows, it is theire bad quality what makes
crackers attack this system. Low fruits, you know?
And there is another thing, that makes linux better: The developers
want to write stable and secure software. It is theire joy and
happiness. They do not mourn, when someone is telling a bug or a
security hole. They are happy, to fix it. Making theire software,
theire "baby" better.
In market, the developers MUST do it, for them fixing software is
just annoying and more work (for the same money). That is the
differnce.
Note: I do not want to claim, linux developers are the better coders.
But they are coding with theire heart. That makes the difference.
It is not the spreading of software.

I accept what you say, the point I was making is that the more users,
and they will be less IT-competent users, the more will login as root.
Windows still makes the first user an administrator, and it takes a bit
of fiddling to set up an unprivileged user and *always* *use* *it*.
It's inconvenient to keep entering the admin password (there's still no
sudo, as far as I know), so people prefer to run with admin privileges.
In most cases, nobody has ever told them why they shouldn't.

This never happens with Linux servers, and not usually with MS ones. I
spent a couple of years on the MS Small Business Server newsgroup,
before it went to web forum, and in every case of a server compromise
it turned out that the admin had been using the web from the server
console, obviously as an administrator. I tried to make this point over
and over, as did the more sensible regular contributors: don't surf the
web with admin privileges, and don't let your users do it.

Basically, I think that with many more users, we would see more Windows
users and they would be less secure in their habits. We've already seen
this to some extent with Ubuntu. I don't think it's any more difficult
to write a virus for Linux than for Windows, but the R number for such
a virus, as epidemiologists would put it, would be very much less than
one, so there's no point. No propagation. I think this would change,
but this is of course just an opinion.

[Nicolas George]

Post by Joe
I accept what you say, the point I was making is that the more users,
and they will be less IT-competent users, the more will login as root.

No, they will not.

And it does not matter, because on a personal computer the root account
is not what matters, what matters is the user account where you can
install a key logger and get banking credentials or encrypt all the data
and ask for a ransom.

Regards,

[Alain D D Williams]

Post by Nicolas George
And it does not matter, because on a personal computer the root account
is not what matters, what matters is the user account where you can
install a key logger and get banking credentials or encrypt all the data
and ask for a ransom.

Which is one of the big problems with MS Windows -- telemetry - which can do
that. Also things like Recall (which only lasted a few weeks recently -
thankfully, but I fear will reappear in some form).

But web browsers are a big problem: Chrome logs all sort of stuff to Google
(but not keystrokes I think), MS Edge does likewise - which is why I stick to
Firefox.

But if you have root access it is easy, I did it on a Unix system V machine in
the late 1980s, a few minutes work. I only needed root as it was for another
user.

[Nicolas George]

I only needed root as it was for another user.

Exactly. On a computer with only one user account, once the pirate have
access to that account, they can do everything that matters. Including
spy the root password next time it is typed, but why waste the time when
everything profitable is already there.

The root account is important for multi-users systems and servers with
privilege separation of services.

Regards,

[Nicholas Geovanis]

Post by Joe
....
Basically, I think that with many more users, we would see more Windows
users and they would be less secure in their habits. We've already seen
this to some extent with Ubuntu. I don't think it's any more difficult
to write a virus for Linux than for Windows, but the R number for such
a virus, as epidemiologists would put it, would be very much less than
one, so there's no point. No propagation. I think this would change,
but this is of course just an opinion.

Linux servers are running headless in data centers, not on many desktops in
comparison. So the desktop set of intrusion vectors are not present on
them. Rarely does a human log into them, they're managed and usually
installed remotely using ansible, salt, CloudFormation on AWS, etc.
Software running on them answers requests at TCP ports, that's what they
do.

[Nicholas Geovanis]

Post by Joe
You missed one: Linux is virtually a virus-free environment, and a
large user base would mean many more people running as root, and it
would become worth the time of malware writers to target Linux. Linux
would become as virus-ridden as Windows.

There is no reason for "many more people running as root" so I don't think
that's a valid point. Ubuntu is derived from Debian and Ubuntu eliminated
direct root login years ago. But you can do that easily with your own
Debian installation if you want to.

It would also become a target for data harvesting, from which Debian,

Post by Joe
at least, is refreshingly free.

Again lacking data center experience? Every server in your data center that
is outward-facing will be contacted by intruders on its open ports. That
includes your Debian servers. If your apache server or application server
running on Debian is vulnerable and open to outside, they will knock on
your door. What happens _after_ that determines how vulnerable you are.

[jeremy ardley]

Post by Nicholas Geovanis
Again lacking data center experience? Every server in your data center
that is outward-facing will be contacted by intruders on its open ports.
That includes your Debian servers. If your apache server or application
server running on Debian is vulnerable and open to outside, they will
knock on your door. What happens _after_ that determines how vulnerable
you are.

A plug for SELinux. It's been around for a long time. It was invented by
the NSA for use by Government agencies but they kindly open sourced it
and it's available on many Distros including Debian.

SELinux is a real pain to get right but when it finally works it's a
tremendous security boost for internet facing systems.

It assumes, correctly, that your internet facing service will be
compromised and the baddy will try to further the exploit. It's
Permissive Action in that unless you specifically permit something to
happen it won't. A web server trying to read any directories that aren't
specified as valid by SELinux will be blocked. A web service trying to
do any system calls not permitted by the policy will be blocked. A web
server trying to send an email will be blocked. etc. etc.

Even better it logs every attempted breach so log monitors can identify
anomalous behaviour in seconds if not milliseconds.

The philosophy of SELinux seems quite different to CrowdStrike

SElinux: "If I don't permit it, it won't happen"

CrowdStrike: "I permit eveything until I get an update to block
something or I suspect something is dodgy"

[Alain D D Williams]

A plug for SELinux. It's been around for a long time. It was invented by the
NSA for use by Government agencies but they kindly open sourced it and it's
available on many Distros including Debian.
SELinux is a real pain to get right but when it finally works it's a
tremendous security boost for internet facing systems.

+1

I use SELinux.

The documentation is awful - there are many different labels that are not
documented as to how they should be used. When there is an issue ausearch will
tell you what to do but not why, I have sometimes found that the recommendation
is wrong and that enabling something else is a better solution.

[Joe]

On Sat, 20 Jul 2024 18:28:09 -0500

Post by Nicholas Geovanis

Post by Joe
You missed one: Linux is virtually a virus-free environment, and a
large user base would mean many more people running as root, and it
would become worth the time of malware writers to target Linux.
Linux would become as virus-ridden as Windows.

There is no reason for "many more people running as root" so I don't
think that's a valid point. Ubuntu is derived from Debian and Ubuntu
eliminated direct root login years ago. But you can do that easily
with your own Debian installation if you want to.

And the important phrase there is 'if you want to'. The point is that
many people, especially those accustomed to running with admin
privileges on their Windows computers, would continue to do that. As
far as I'm concerned, the owner of a computer *must* have admin rights
to that computer, but *must* use those rights carefully and only when
necessary, and absolutely never use a web browser or read email with
those rights enabled.

Post by Nicholas Geovanis
It would also become a target for data harvesting, from which Debian,

Post by Joe
at least, is refreshingly free.

Again lacking data center experience? Every server in your data
center that is outward-facing will be contacted by intruders on its
open ports. That includes your Debian servers. If your apache server
or application server running on Debian is vulnerable and open to
outside, they will knock on your door. What happens _after_ that
determines how vulnerable you are.

I wasn't referring to 'genuine' malware, but that which is included in
the OS itself at manufacture. The recent versions of Windows include
more 'telemetry' than before, which you can allegedly disable. How much
information about *your* use of *your* computer do you think belongs to
the OS vendor? I would say "none at all", and I would not trust for a
moment "OK, we promise not to look if you tick this little box here".
MS has for a long time made it difficult to even login to your own
computer without also logging in to an MS account. So far, it is still
avoidable, just about.

[Nicolas George]

Post by Joe
And the important phrase there is 'if you want to'. The point is that
many people, especially those accustomed to running with admin
privileges on their Windows computers, would continue to do that.

<sigh> No, they will not. They will continue to follow the system
default, whatever it is.

And once again, this is a waste of time because being root is not what
matters on a personal computer.

[gene heskett]

Post by Hans
Hello,
well, the thing is: Do we really want to go to more market share?
Let's imagine, Debian becomes market relevant, what will happen? Sure, more
developers get paid, what is very nice. But not all developers will.
Many good developers will not be paid and when the market will rule things,
then many good developers will be pushed away or demoralied. Because it will
become common, that people will no more cherish theire work.
The development of a few people will be cherished, those, who create programs,
the market wants.
I am using linux since more than 30 years and it is impressive, what people
can do, when they can do, what they want and what they like.
And look at the quality, look, what has been created since the beginning. This
was only possible, because no market forced people, to do things the market
wants, not what the developers want.
I think, we all can be happy, that we are not dependent from any market, the
developers, because theire freedom and theire contentement is not been
deminished, and the users, who get very good and high qulitative software to
work with.
And if you really think, the more you spend, the better the software, you can
of course buy software only from the market.
Or, you can donate linux developers and/or distributors of your money.
Personally(!) I think, the second way is better, because I can speak directly
to developers, could (if I would be capable of) fix things myself together
with the developers and maybe can even ask him, to implenent some functions
especially for me.
All things, a market driven software will never offer.
So, I think, we can be happy, that linux (and debian) is not market relevant.
It will lose its freedom, its high quality and the joy of many people.
Sorry, if I did not always find the right expression, I am not native English.

And even you Hans, leave out the major, all encompassing, reason for the
lack of market share, which is that most business that have a
computerized system to run things also value what their MBA says. And
since there is no one to sue to cover their personal butt in case the
system goes south like cloudflare has in the last 3 days, M$ &
cloudflare are a brick and morter legal target they can sic the legal
team onto.

Their is essentially no one in the linux arena to sue if things go
south, so it doesn't take more than an eighth grade education to see why
they won't ever recommend linux no matter how superior it may be at the
end of a P&L report. They have to have someone to sue. Bill Shakespear
said it best when he wrote "first, we kill all the lawyers." But MBA's
had not yet crawled out of the slime schools yet, so he can't be blamed
for not including MBA's when he wrote that famous phrase.

Post by Hans
Best regards
Hans
.

Cheers, Gene Heskett, CET.

[t***@tuxteam.de]

On Sat, Jul 20, 2024 at 03:27:17PM -0400, gene heskett wrote:

[...]

Post by gene heskett
And even you Hans, leave out the major, all encompassing, reason for the
lack of market share, which is that most business that have a computerized
system to run things also value what their MBA says. And since there is no
one to sue to cover their personal butt in case the system goes south like
cloudflare has in the last 3 days, M$ & cloudflare are a brick and morter
legal target they can sic the legal team onto.

First: it wasn't cloudflare -- it was CrowdStrike (a sec firm, of all
things!)

Second: nobody's going to sue them. Guess what? The big ones have lawyers,
lots of them. And their best protected tech is "law tech". They wouldn't
be skimping on quality if it didn't pay off.

Case in point: Solarwinds. 2020, they had a row of high-level attacks
which knocked off their customer's customers (AFAIR, one third of
Sweden's supermarkets had to close for three to four days, among many
other things).

They were sued for $26 million, that's it.

Cheers

[1] https://en.wikipedia.org/wiki/SolarWinds#2019%E2%80%932020_supply_chain_attacks

[Nicholas Geovanis]

Post by gene heskett
[...]

Post by gene heskett
And even you Hans, leave out the major, all encompassing, reason for the
lack of market share, which is that most business that have a

computerized

Post by gene heskett
system to run things also value what their MBA says. And since there is

no

Post by gene heskett
one to sue to cover their personal butt in case the system goes south

like

Post by gene heskett
cloudflare has in the last 3 days, M$ & cloudflare are a brick and morter
legal target they can sic the legal team onto.

First: it wasn't cloudflare -- it was CrowdStrike (a sec firm, of all
things!)
Second: nobody's going to sue them. Guess what? The big ones have lawyers,
lots of them. And their best protected tech is "law tech". They wouldn't
be skimping on quality if it didn't pay off.
Case in point: Solarwinds. 2020, they had a row of high-level attacks
which knocked off their customer's customers (AFAIR, one third of
Sweden's supermarkets had to close for three to four days, among many
other things).
They were sued for $26 million, that's it.

Every time I meet or work for someone who is still running SolarWinds
products (many many :-) I remind them of this: SolarWinds' source-code
repositories were broken into, the source-code modified by the intruders,
and their changes checked back in like good software developers :-) Then
the corporation sent you that software and you paid for it.

How do you feel? Suppose that the same thing was done to the software in
your car? Would you drive it again? Or in the aircraft you will fly-in next
month? Would you take that plane?

Cheers

Post by gene heskett
[1]
https://en.wikipedia.org/wiki/SolarWinds#2019%E2%80%932020_supply_chain_attacks
--
t

[Joe]

On Sat, 20 Jul 2024 15:27:17 -0400

Post by gene heskett
And even you Hans, leave out the major, all encompassing, reason for
the lack of market share, which is that most business that have a
computerized system to run things also value what their MBA says.
And since there is no one to sue to cover their personal butt in case
the system goes south like cloudflare has in the last 3 days, M$ &
cloudflare are a brick and morter legal target they can sic the legal
team onto.
Their is essentially no one in the linux arena to sue if things go
south, so it doesn't take more than an eighth grade education to see
why they won't ever recommend linux no matter how superior it may be
at the end of a P&L report. They have to have someone to sue. Bill
Shakespear said it best when he wrote "first, we kill all the
lawyers." But MBA's had not yet crawled out of the slime schools yet,
so he can't be blamed for not including MBA's when he wrote that
famous phrase.

It's a little bit more subtle than that. Debian offers exactly the same
software warranty as MS or CloudStrike i.e. zilch. Larger businesses
generally buy service contracts from middlemen, who are the ones who
get sued. And so they should be if they have not provided, as part of
their contract, quick and reliable recovery systems, and immediate
response to emergency calls.

Overnight full backups would have solved this problem, and it would
never have arisen if the system admins had disabled automatic updates
and waited the customary few days before applying them manually, to see
how many people screamed on the day of release. Quite a few, in this
case.

I think that thought of legal action is fairly low on the list of
someone bleeding a million pounds for every hour that their system is
down, who primarily want immediate and effective help to get running
again. When the dust settles, the company accountants will go looking
for someone to blame.

It is indeed backup when things go bad that Open Source software is
definitely lacking, but it's the overall system administration and fast
response time that is the problem, not the software itself, which never
carries warranty no matter how much has been paid for it. If a business
chooses Linux for its IT work, it must do so via a Linux service
business that will provide the necessary service level agreement, but
that is exactly the same position that Windows users are in.

MS, if you have done no more than buy a server OS and install it
yourself, will provide free, best efforts telephone/email help if a
server is down. But that's generally not going to be enough to get
running quickly, especially if you've been skimping on backups.

The biggest problem that Linux (and Mac systems) has is that people are
programmed early. Windows computers are used in schools and most
universities. Computer software training courses are based on Windows.
And so on. In the early 1990s, BBC Micros were being replaced in UK
schools, mostly with early IBM compatible PCs running Windows 3. In
vain did Acorn try to sell them Archimedes computers (running on ARM2
or ARM3, by the way). "But when they leave school, they will need to be
familiar with Windows", said the education authorities. Of course, when
the pupils left school, what they needed to be familiar with was Windows
95, which bore a much closer resemblance to RiscOS on the 32-bit
Archimedes than it did to Windows 3.

About 8 years ago I assisted a team of Japanese engineers to do some
retrofit work on a number of already-delivered trains. The train
operating system was Linux, not Debian, and they were a bit secretive
about it, but I think it was Fedora. They were amazed to find I was
running Linux on my netbook, and said they had never seen Linux used on
a workstation. These were fairly bright people, not all young, working
for a large company.

[The Wanderer]

On Sat, 20 Jul 2024 15:27:17 -0400 gene heskett

Post by gene heskett
And even you Hans, leave out the major, all encompassing, reason
for the lack of market share, which is that most business that have
a computerized system to run things also value what their MBA
says. And since there is no one to sue to cover their personal butt
in case the system goes south like cloudflare has in the last 3
days, M$ & cloudflare are a brick and morter legal target they can
sic the legal team onto.
Their is essentially no one in the linux arena to sue if things go
south, so it doesn't take more than an eighth grade education to
see why they won't ever recommend linux no matter how superior it
may be at the end of a P&L report. They have to have someone to
sue. Bill Shakespear said it best when he wrote "first, we kill
all the lawyers." But MBA's had not yet crawled out of the slime
schools yet, so he can't be blamed for not including MBA's when he
wrote that famous phrase.

It's a little bit more subtle than that. Debian offers exactly the
same software warranty as MS or CloudStrike i.e. zilch. Larger
businesses generally buy service contracts from middlemen, who are
the ones who get sued. And so they should be if they have not
provided, as part of their contract, quick and reliable recovery
systems, and immediate response to emergency calls.
Overnight full backups would have solved this problem,

How? That is, how would they have eliminated the need to go touch each
computer in order to get it reverted to a state where it can be managed
by e.g. the systems which could restore from the most recent backup?

and it would never have arisen if the system admins had disabled
automatic updates and waited the customary few days before applying
them manually, to see how many people screamed on the day of
release. Quite a few, in this case.

While I agree that the admins of the CrowdStrike backend systems should
have done more testing before releasing this update to be deployed to
client endpoints in the wild, I have no reason to think that that
release is controlled by an "automatic updates" mechanism, nor that it
is the type of update which it is customary to wait before releasing.

For the admins of the endpoint systems which are running the CrowdStrike
Falcon sensor, it really depends on which kind of update this was. If
this was a new version of the sensor software itself, then there is
indeed a delay mechanism available, and in fact built in to the control
console for the software, and I fully expect that most people who
administer the software for the client enterprises are taking advantage
of it.

That new-version-delay mechanism lets sysadmins divide their endpoints
into groups, and decide which sensor version each group will run: the
latest, the next-to-latest, or the one before that. (You can even move
endpoints from one group to another, and see them change versions - even
potentially downgrading - within short order.) At my own workplace, we
have nearly everything set to "the one before that", i.e. two versions
prior to the current release - exactly in order to avoid being hit by
problems like this one.

In this case, however, the problematic update appears to have gone out
to *all sensor versions simultaneously*.

That tells me that rather than being an update to the sensor itself,
this almost has to have been an update to the *data files* used by the
sensor as it operates - the equivalent of a definition update, for other
common antivirus-type tools. With most such tools that I'm aware of,
those type of updates are typically released *daily*, and being even one
day behind can leave you vulnerable to a zero-day exploit.

I am not at all certain that there is any mechanism to disable
"automatic update" of that type of data, or even that there *should* be;
I am certainly not aware of any customary practice of waiting a few days
before deploying that type of update. Even if there is such a mechanism
and such a practice, the frequent releases and the potentially high
impact of a delay would seem to make it unreasonable for sysadmins to be
expected to make use of them.

(I've snipped the rest of what you wrote, as I have no particular
disagreement with any of it, and agree with some in ways that I don't
feel the need to express.)

[Russell L. Harris]

The same reasons the standard typewriter keyboard is QWERTY rather
than Dvorak:

= The precedent set by the first to market is powerful.

= The influence of advertising upon a populace lacking in discernment
and addicted to novelty is deadly.

Add to that extortion and bribes and a compromised legal system.

The QWERTY system was designed to slow down typists so as to reduce
the problem of jamming of keys of a poorly-designed mechanism.

Much of the evil in the world is due to the unbridled pursuit of
money:

For the love of money is the root of all sort of evil: which while some
coveted after, they have erred from the faith, and pierced themselves
through with many sorrows. - I Timothy 6:10

RLH

[Nicolas George]

is it possible to remap keyboard to Dvorak in X Window?

Yes, of course.

https://letmegooglethat.com/?q=Debian+dvorak

does anyone use it
to speed up typing?

No, only to feel smug.

# Later experiments have shown that many keyboard designs, including some
# alphabetical ones, allow very similar typing speeds to QWERTY and Dvorak
# when typists have been trained for them, suggesting that Dvorak's
# careful design principles may have had little effect because keyboard
# layout is only a small part of the complicated physical activity of
# typing.[19]

https://en.wikipedia.org/wiki/Dvorak_keyboard_layout

Regards,

[George at Clug]

Contrary to popular belief, the QWERTY layout was not designed to slow the typist down, but rather to speed up typing.

Indeed, there is evidence that, aside from the issue of jamming, placing often-used keys farther apart increases typing speed, because it encourages alternation between the hands.

https://en.wikipedia.org/wiki/QWERTY

Post by Nicolas George

is it possible to remap keyboard to Dvorak in X Window?

Yes, of course.
https://letmegooglethat.com/?q=Debian+dvorak

does anyone use it
to speed up typing?

No, only to feel smug.
# Later experiments have shown that many keyboard designs, including some
# alphabetical ones, allow very similar typing speeds to QWERTY and Dvorak
# when typists have been trained for them, suggesting that Dvorak's
# careful design principles may have had little effect because keyboard
# layout is only a small part of the complicated physical activity of
# typing.[19]
https://en.wikipedia.org/wiki/Dvorak_keyboard_layout
Regards,
--
Nicolas George

[Russell L. Harris]

Post by Russell L. Harris
The same reasons the standard typewriter keyboard is QWERTY rather
= The precedent set by the first to market is powerful.
= The influence of advertising upon a populace lacking in discernment
and addicted to novelty is deadly.
Add to that extortion and bribes and a compromised legal system.
The QWERTY system was designed to slow down typists so as to reduce
the problem of jamming of keys of a poorly-designed mechanism.

is it possible to remap keyboard to??Dvorak in X Window? does anyone
use it to speed up typing?

ISO published a Dvorak standard, but it was compromised, for the top
(numeric) rows of keys were in the order 1234567890 . Dvorak has the
keys in the order 7531902468 .

For several years now, Debian has offered both the bastardized ISO
mapping ("US > Dvorak") and the original Dvorak arrangement ("US >
Dvorak Classic").

IBM manufactured a Selectric with the ISO Dvorak keyboard. This was
the original Selectric, not the Correcting Selectric II.

Back when I ran Window$, one or two keyboard manufacturers (I seem to
recall the name "Northgate") made Dvorak keyboards and even included a
set of Dvorak keycaps.

For me, a Macintosh guru changed the key mapping on a MacClassic to
Dvorak.

And long ago in Debian, with a bit of help, I managed to change the
key mapping file to Dvorak.

When in High School (A.D. 1963) I learned to type (QWERTY), the
typewriters in the classroom had blank keycaps. A layout chart was
hung on the wall in the front of the room. We learned to "touch
type," and were able to reach 95 words per minute.

I switched to Dvorak circa A.D. 1985, when I was given a project which
required much typing. I made learning Dvorak a matter of "swim or
sink." The first couple of weeks were painful, but within a month all
was well.

And when touch-typing, the labels on the keycaps do not matter. All
my keyboards are standard QWERTY.

In an office environment, the guy using Dvorak with a keyboard labeled
QWERTY has no worries about others messing with his computer.

RLH

[Russell L. Harris]

Your final statement makes me curious about learning Dvorak.
Shawn Jefferds
??n ??f?rdz
Noli fovere canem ardentum
Vote Vader 2024!

Post by Russell L. Harris
The same reasons the standard typewriter keyboard is QWERTY rather
= The precedent set by the first to market is powerful.
= The influence of advertising upon a populace lacking in discernment
and addicted to novelty is deadly.
Add to that extortion and bribes and a compromised legal system.
The QWERTY system was designed to slow down typists so as to reduce
the problem of jamming of keys of a poorly-designed mechanism.

is it possible to remap keyboard to??Dvorak in X Window? does anyone
use it to speed up typing?

ISO published a Dvorak standard, but it was compromised, for the top
(numeric) rows of keys were in the order 1234567890 . Dvorak has the
keys in the order 7531902468 .
For several years now, Debian has offered both the bastardized ISO
mapping ("US > Dvorak") and the original Dvorak arrangement ("US >
Dvorak Classic").
IBM manufactured a Selectric with the ISO Dvorak keyboard. This was
the original Selectric, not the Correcting Selectric II.
Back when I ran Window$, one or two keyboard manufacturers (I seem to
recall the name "Northgate") made Dvorak keyboards and even included a
set of Dvorak keycaps.
For me, a Macintosh guru changed the key mapping on a MacClassic to
Dvorak.
And long ago in Debian, with a bit of help, I managed to change the
key mapping file to Dvorak.
When in High School (A.D. 1963) I learned to type (QWERTY), the
typewriters in the classroom had blank keycaps. A layout chart was
hung on the wall in the front of the room. We learned to "touch
type," and were able to reach 95 words per minute.
I switched to Dvorak circa A.D. 1985, when I was given a project which
required much typing. I made learning Dvorak a matter of "swim or
sink." The first couple of weeks were painful, but within a month all
was well.
And when touch-typing, the labels on the keycaps do not matter. All
my keyboards are standard QWERTY.
In an office environment, the guy using Dvorak with a keyboard labeled
QWERTY has no worries about others messing with his computer.
RLH

I always was a good typist, but before switching to Dvorak, I hated to
type numeric material. But with the original Dvorak layout (in
Debian, Dvorak Classic) numbers are a joy.

Of course, with recent Debian systems, the keyboard mapping can
automatically change depending on the user, in which case the login
screen ought to be QWERTY; see SETTINGS MANAGER > KEYBOARD.

RLH

[songbird]

Russell L. Harris wrote:
...

Post by Russell L. Harris
When in High School (A.D. 1963) I learned to type (QWERTY), the
typewriters in the classroom had blank keycaps. A layout chart was
hung on the wall in the front of the room. We learned to "touch
type," and were able to reach 95 words per minute.

i flunked typing in Jr. High school, i had a problem with
the teacher but also hated manual typewriters. little did i
know that a handful of years later i'd be spending many hours
in a computer lab typing on keypunches and other layouts.

i ended up learning how to touch type, but it did take some
time. but i did learn. i'm sure i've got my ten thousand
hours or more by now.


...

Post by Russell L. Harris
And when touch-typing, the labels on the keycaps do not matter. All
my keyboards are standard QWERTY.

true, i'm typing on a keyboard that has no visible labels
on the keys unless i turn on the LED lights which shine
through the keys to light them up. it helps that there are
the bumps on the home row keys and the number 5 on the
number pad.

Post by Russell L. Harris
In an office environment, the guy using Dvorak with a keyboard labeled
QWERTY has no worries about others messing with his computer.

:)


songbird

[Andy Smith]

Hi,

crowdstrike makes news headlines, many Windows become blue screens
it is evident that many people around still use Windows
i wonder if linux is more reliable than Windows

For this specific issue, if Linux were used at the same scale and
for the same purposes as these affected Windows machines, then a
similar issue would affect Linux sooner or later.

The reason why this is the case is that the current motivation for
the use of Crowdstrike's software on those Windows machines would
be exactly the same if they were Linux machines, and so these
companies would do the same thing with the same end result.

In fact, Crowdstrike already made a similar mistake earlier this
year with one of their Linux solutions which resulted in end user
machines having a kernel panic. Debian stable end user machines. So
there is no practical difference between Crowdstrike+Windows and
Crowdstrike+Linux.

https://news.ycombinator.com/item?id=41005936

So then you might assume that the problem here is Crowdstrike's
incompetence and a better vendor would solve all problems. You would
be wrong, because the world is full to the brim with inept software
vendors and there is no real consequence for software failures.

I expect Crowdstrike's stock value to recover and for this incident
to be forgotten, but even if it isn't it doesn't really matter
because there is an infinite line of similar companies to step into
their clown shoes.

The state of the software supply chain on Linux is not any better
than on Windows, and it may even be worse. You don't notice because
Linux is extremely niche for everything but Internet services and we
don't often look outside our bubble.

We have nothing to be smug about.

To be clear I would never run anything like Crowdstrike on any
machine I had authority over, but my opinion does not change the
fact that demonstrably the majority of the market thinks
and acts differently. This event will not change that, either, but
if you had said, "people need to stop running software like this"
instead of "people need to run Linux", I would be able to agree with
you. Just saying "we need better software" isn't a very catchy
polemic though is it.

Thanks,
Andy

[d***@howorth.org.uk]

Post by Andy Smith
Hi,

crowdstrike makes news headlines, many Windows become blue screens
it is evident that many people around still use Windows
i wonder if linux is more reliable than Windows

For this specific issue, if Linux were used at the same scale and
for the same purposes as these affected Windows machines, then a
similar issue would affect Linux sooner or later.
The reason why this is the case is that the current motivation for
the use of Crowdstrike's software on those Windows machines would
be exactly the same if they were Linux machines, and so these
companies would do the same thing with the same end result.
In fact, Crowdstrike already made a similar mistake earlier this
year with one of their Linux solutions which resulted in end user
machines having a kernel panic. Debian stable end user machines. So
there is no practical difference between Crowdstrike+Windows and
Crowdstrike+Linux.
https://news.ycombinator.com/item?id=41005936
So then you might assume that the problem here is Crowdstrike's
incompetence and a better vendor would solve all problems. You would
be wrong, because the world is full to the brim with inept software
vendors and there is no real consequence for software failures.

It seems clear to me that what's needed is a change in the law. At the
moment here in the UK we have national news services explaining that
airline passengers won't be able to get compensation because the
'event' was outside the airline's control. That's clearly nonsense
since some airlines weren't affected so perhaps sense will eventually
prevail and the companies that have had problems will be held liable
for damages to their customers. But it would be better if they could
then sue Crowdstrike for installing the faulty update. (Perhaps they
can? I don't know, IANAL.) That might provide some incentive to improve
the systems and processes so problems like this don't occur again.

[Andy Smith]

Hi,

Post by d***@howorth.org.uk
It seems clear to me that what's needed is a change in the law. At the
moment here in the UK we have national news services explaining that
airline passengers won't be able to get compensation because the
'event' was outside the airline's control. That's clearly nonsense
since some airlines weren't affected so perhaps sense will eventually
prevail and the companies that have had problems will be held liable
for damages to their customers.

And yes here in the UK where we allowed the Post Office to pay
billions to Fujitsu to develop the Horizon IT system that
incorrectly accused hundreds of postmasters of fraud, resulting in
criminal prosecutions and at least one case of suicide.

Innocent people died and went to jail — lives were ended and ruined
— and there will be no real consequences for those people to blame.
We will be lucky to see any criminal prosecution of Post Office
management, if there are any they will be a joke, and absolutely
nothing will happen to the vendor Fujitsu UK.

There is still nothing stopping a Horizon IT incident on Linux.

So yes, agreed, the software industry needs to grow up and it's
pointless arguing for our tribe within it at this level.

Thanks,
Andy

[Nicolas George]

Post by Andy Smith
And yes here in the UK where we allowed the Post Office to pay
billions to Fujitsu to develop the Horizon IT system that
incorrectly accused hundreds of postmasters of fraud, resulting in
criminal prosecutions and at least one case of suicide.

That was not a bug, that was a feature.

This kind of thing happens not because the industry is clumsy: all
industries are somewhat clumsy.

This kind of thing happens because politicians are perfectly to let a
clumsy industry handle people' lives. The scope statement probably
insisted more in avoiding false negatives than false positives.

Regards,

[gene heskett]

Post by d***@howorth.org.uk

Post by Andy Smith
Hi,

crowdstrike makes news headlines, many Windows become blue screens
it is evident that many people around still use Windows
i wonder if linux is more reliable than Windows

For this specific issue, if Linux were used at the same scale and
for the same purposes as these affected Windows machines, then a
similar issue would affect Linux sooner or later.
The reason why this is the case is that the current motivation for
the use of Crowdstrike's software on those Windows machines would
be exactly the same if they were Linux machines, and so these
companies would do the same thing with the same end result.
In fact, Crowdstrike already made a similar mistake earlier this
year with one of their Linux solutions which resulted in end user
machines having a kernel panic. Debian stable end user machines. So
there is no practical difference between Crowdstrike+Windows and
Crowdstrike+Linux.
https://news.ycombinator.com/item?id=41005936
So then you might assume that the problem here is Crowdstrike's
incompetence and a better vendor would solve all problems. You would
be wrong, because the world is full to the brim with inept software
vendors and there is no real consequence for software failures.

It seems clear to me that what's needed is a change in the law. At the
moment here in the UK we have national news services explaining that
airline passengers won't be able to get compensation because the
'event' was outside the airline's control. That's clearly nonsense
since some airlines weren't affected so perhaps sense will eventually
prevail and the companies that have had problems will be held liable
for damages to their customers. But it would be better if they could
then sue Crowdstrike for installing the faulty update. (Perhaps they
can? I don't know, IANAL.) That might provide some incentive to improve
the systems and processes so problems like this don't occur again.
.

That bit of legaleze should have been addressed about the time NT3.51
came out. Maybe by now M$ would have been stung in the bank balance
enough to have learned they will get caught out eventually. NT deleted
the main OS library, and of coarse would not boot. I put the drive in
another machine and poked around a bit, finally finding a file that was
apparently part of the drives housekeeping but only called if a call to
rand returned a certain date in the future which turned out to be about
a day in the past. But it contained nothing in the way of a check to see
if the file belonged to the os. I called support, but had no
registration for that copy because it was a bulk purchase by the
network, and all the tv stations got was the machine pre-installed, the
network had not given us the paper work. So I explained to M$ support
and got called a pie rat by support. Screw M$ and the camel that rode in
on them. I packed the drive in a padded box & handed it to the fedex
driver. The network net guru reinstalled and overnighted it back. But
while it was down, the lack of data to program our 7 meter C band dish
cost us about 5k$ a day because we were not airing the commercials we
were contracted to transmit.

So now you know why my hatred of M$ is very long term and incurable.

Cheers, Gene Heskett, CET.

[Anssi Saari]

why free OS hasn't gained more share even after 30 years of development?

But it has. The internet and what connects to it now mostly run Linux,
other than Microsoft's single niche. Mobile phones run a Linux
variant. The PC desktop is the only exception where they have
domination, anywhere else MS is an also-ran or nothing.

Microsoft was recently in near panic since they have nothing on mobile
and their main business (Windows + Office on PC desktops) is
shrinking. I guess they managed to compensate by becoming a cloud player
with Azure. Selling virtualized Linux now.

[t***@tuxteam.de]

On Tue, Jul 30, 2024 at 10:44:37AM +0800, hlyg wrote:

[...]

PS: i am aware that linux has more success in server market

... and the mobile market. Android is, on its underbelly, Linux
after all. So Linux might have the most installations out there,
I guess.

Not that Microsoft didn't try -- they even bought one big phone
manufacturer (Nokia) and killed [1] it in the process of trying
to ram Winphone down the people's throats (Sony paid its price
too). They failed miserably.

The downside of all of that is that it took another monster of
surveillance capitalism to float Linux on that platform, and that
this Linux is unfree in many other strange ways.

It's capitalism: it takes money to make money.

Cheers

[1] See Steven Elop if you want to have some spectacular corporate
drama: https://en.wikipedia.org/wiki/Steven_Elop#CEO_of_Nokia

[John Hasler]

Children are taught in elementary school that computer == Windows.