How to update environment variable output

[Keith Bainbridge]

Addendum

So I tried opening a new xterm tab and ran


Tue ***@17:07:43 205.2024 AEST
:~ $> mkcd /tmp/$DOYR.$YEAR
and landed in /tmp/205.2024 $>

Looking good

From the tab I had used earlier, ran source .bashrc

then
:/tmp/205.2024 $> mkcd /tmp/day$DOYR.$YEAR

and landed in
:/tmp/day205.2024 $> again, just as expected.


So, now question is how can I get cron to 'source .bashrc' I'm
thinking add that line to my script?





Good afternoon All

For reference, today is Tue ***@15:41:47 205.2024 AEST

This is part of my command prompt, generated by

PS1='\n \u@\h \n\n $(date +"%a %d%b%Y@%H:%M:%S %j.%Y %Z") \n :\w $> '

My calculation is that today is day 205

When I run this function


mkcd ()
{
mkdir -p $1
cd $1
}
in the form :~ $> mkcd
/mnt/data/keith/Documents/$YEAR/$MTH$YEAR/$DOYR.$YEAR/

I am transferred to
/mnt/data/keith/Documents/2024/Jul2024/196.2024 $> ie $DOYR is
using the wrong day number

So I cd to my home and run the commands separately, in the form

:~ $> mkdir -p /mnt/data/keith/Documents/$YEAR/$MTH$YEAR/$DOYR.$YEAR/

***@lenv0

Tue ***@15:38:24 205.2024 AEST
:~ $> cd /mnt/data/keith/Documents/$YEAR/$MTH$YEAR/$DOYR.$YEAR/

and land in /mnt/data/keith/Documents/2024/Jul2024/196.2024

ls -lah /mnt/data/keith/Documents/2024/Jul2024
total 104K
drwxr-xr-x 8 keith keith 4.0K Jul 22 13:33 .
drwxr-xr-x 30 keith keith 4.0K Jul 13 11:53 ..
sdrwxr-xr-x 2 keith keith 4.0K Jul 22 13:33 196.2024
drwxr-xr-x 2 keith keith 4.0K Jul 13 11:53 day189.2024
drwxr-xr-x 2 keith keith 4.0K Jul 13 11:53 day192.2024
drwxr-xr-x 2 keith keith 4.0K Jul 13 11:53 day193.2024
drwxr-xr-x 2 keith keith 4.0K Jul 13 11:53 day194.2024
drwxr-xr-x 2 keith keith 4.0K Jul 22 13:21 day196.2024

shows that directory 196.2024 was created yesterday. (My original
format for the new dir was $MTH$YEAR/day$DOYR.$YEAR/ I took the 'day'
out in case that was creating a problem that I haven't seen before.


OK then, I have 2 questions:

Why is the env var $DOYR not updating when I use it at the command
prompt AND for that matter in a script - clearly the system is or I'd be
getting the wrong day # at my command prompt?

Why am I not getting a warning that the dir already exists?

I want to run the script as a cron job, to keep a daily copy of a few
files - mainly to prove that I can do it.

Thanks for any suggestions

By the bye, I expanded the date format for my command prompt as part of
the discussion I started a few weeks back about the date the sender sent
mail showing in a reply.
--
All the best

Keith Bainbridge

***@gmail.com
***@gmail.com
[phone]

UTC + 10:00

[Greg Wooledge]

Post by Keith Bainbridge
From the tab I had used earlier, ran source .bashrc
then
:/tmp/205.2024 $> mkcd /tmp/day$DOYR.$YEAR

So you're setting those variables one time inside your .bashrc file?

This is quite bad. What happens when you have a terminal window that
you open before midnight, and then continue using *after* midnight?
Those variables will still be set to the previous day's values.

Also, why did you start three separate threads for this question?

Post by Keith Bainbridge
So, now question is how can I get cron to 'source .bashrc' I'm thinking
add that line to my script?

I would advise against it. Who knows what kind of silly things your
.bashrc may do that you wouldn't want an automated script to pick up.
Even if you verify that your .bashrc is "cron safe" right now, you
might add something to it a year from now, forgetting that you need
it to remain "cron safe", and accidentally break the script.

Any command that's supposed to act based on the current day will need
to perform its *own* request to fetch the current day from the system
clock. It cannot rely on an environment variable set some unknown
length of time in the past.

Use the date command or bash's printf %()T inside your bash script to
get the current date.

Moreover, never make *two* calls to the system clock in the same script.
If you need the Julian day number (%j) and also the calendar
day-of-the-month number (%d), don't do this:

# WRONG
julian=$(date +%j)
dom=$(date +%d)

That calls out to the system clock twice, and gets two different values.
If the script is executed right at midnight, you might get two different
days.

Instead, only make *one* call. There are a couple different ways to do
this. One is to get all the date(1) fields you need at once, and then
parse them out:

read -r julian dom < <(date "+%j %d")

Another is to fetch the epoch time value (%s) and then use that value
in all future calls. With GNU date:

now=$(date +%s)
julian=$(date -d "@$now" +%j)
dom=$(date -d "@$now" +%d)

Or with bash's printf, on Linux or BSD:

printf -v now '%(%s)T' -1
printf -v julian '%(%j)T' "$now"
printf -v dom '%(%d)T' "$now"

(Bash 5.0 added an EPOCHSECONDS variable as well.)

This same concept applies in any programming language you use. It's not
just a shell script thing, even though I'm showing shell script examples
here.

[Keith Bainbridge]

Post by Greg Wooledge

Post by Keith Bainbridge
From the tab I had used earlier, ran source .bashrc
then
:/tmp/205.2024 $> mkcd /tmp/day$DOYR.$YEAR

So you're setting those variables one time inside your .bashrc file?

The variables are in .bashrc. I believed that was a good place to keep them. I'll check the original tomorrow, but it looks like a couple are there by default


I intended to list all variables in my original note. Seems I need to put them else where

Post by Greg Wooledge
This is quite bad. What happens when you have a terminal window that
you open before midnight, and then continue using *after* midnight?
Those variables will still be set to the previous day's values.

The day# in my command prompt increments when I start in the morning. Maybe I need to press enter. I'll check tomorrow and confirm

Post by Greg Wooledge
Also, why did you start three separate threads for this question?

I wasn't aware I had started new threads. I realised as soon as I read that comment, that all I had to do was reply to group with the addenda

Post by Greg Wooledge

Post by Keith Bainbridge
So, now question is how can I get cron to 'source .bashrc' I'm thinking
add that line to my script?

I would advise against it. Who knows what kind of silly things your
.bashrc may do that you wouldn't want an automated script to pick up.
Even if you verify that your .bashrc is "cron safe" right now, you
might add something to it a year from now, forgetting that you need
it to remain "cron safe", and accidentally break the script.
Any command that's supposed to act based on the current day will need
to perform its *own* request to fetch the current day from the system
clock. It cannot rely on an environment variable set some unknown
length of time in the past.
Use the date command or bash's printf %()T inside your bash script to
get the current date.

Right ooo

Post by Greg Wooledge
Moreover, never make *two* calls to the system clock in the same script.
If you need the Julian day number (%j) and also the calendar
# WRONG
julian=$(date +%j)
dom=$(date +%d)
That calls out to the system clock twice, and gets two different values.
If the script is executed right at midnight, you might get two different
days.
Instead, only make *one* call. There are a couple different ways to do
this. One is to get all the date(1) fields you need at once, and then
read -r julian dom < <(date "+%j %d")
Another is to fetch the epoch time value (%s) and then use that value
now=$(date +%s)
printf -v now '%(%s)T' -1
printf -v julian '%(%j)T' "$now"
printf -v dom '%(%d)T' "$now"
(Bash 5.0 added an EPOCHSECONDS variable as well.)
This same concept applies in any programming language you use. It's not
just a shell script thing, even though I'm showing shell script examples
here.

Just quickly, from my tablet



More in the morning.
--
All the best

Keith BAINBRIDGE

[phone]
***@gmail.com
***@mail.com

GMT + 10
From my Apad

[Greg Wooledge]

Post by Keith Bainbridge
The day# in my command prompt increments when I start in the morning. Maybe I need to press enter.

That makes it sound like you're setting the YEAR et al. variables in the
PROMPT_COMMAND variable.

If that's the case, it's *less* wrong, but only a little bit. You still
have the issue that the date might change while you're sitting at a
stale shell prompt from yesterday, with stale date/time variables.

Really, the fundamental problem here is that you should not be storing
date/time information in long-lived variables, and then trying to use
those variables at an indeterminate point in the future.

Anything that needs to act upon the current date should fetch the current
date immediately before acting.

[David Wright]

Post by Greg Wooledge

Post by Keith Bainbridge
The day# in my command prompt increments when I start in the morning. Maybe I need to press enter.

That makes it sound like you're setting the YEAR et al. variables in the
PROMPT_COMMAND variable.
If that's the case, it's *less* wrong, but only a little bit. You still
have the issue that the date might change while you're sitting at a
stale shell prompt from yesterday, with stale date/time variables.

Actually, that is what I do want (the time—the date not so much).
That tells you when the last command finished. Press Return before
you start a command and you get some idea of how long it takes
to run. For the current time, press Return.

I think putting the time±date in one's prompt is pretty popular,
judging by the number of ways of specifying it are listed in
man bash under PROMPTING. As well as employing those special
characters for the time elements, you can use the format specifiers
from the date command if you prefer: just wrap then in the \D{…}
special "character".

I use both myself: just the 24-hour time in xterms (where I have
xclock showing "Tue 23" and a swissclock for good measure), but
in VCs I include Tue23 because there's no other calendar reminder.

case "$TERM" in
*linux*)
PS1+='\H!\u \D{%a%d %T} \w\$ \['$(tput sgr0)'\]'
;;
*)
PS1+='\H!\u \t \w\$ \['$(tput sgr0)'\]'
;;
esac

(Note: the += is because my PS1 already has colour and error code
information within it. Also, confusingly, %T and \t both yield
24-hour times in their respective positions.)

Post by Greg Wooledge
Really, the fundamental problem here is that you should not be storing
date/time information in long-lived variables, and then trying to use
those variables at an indeterminate point in the future.
Anything that needs to act upon the current date should fetch the current
date immediately before acting.

Cheers,
David.

[Greg Wooledge]

Post by David Wright

Post by Greg Wooledge

Post by Keith Bainbridge
The day# in my command prompt increments when I start in the morning. Maybe I need to press enter.

That makes it sound like you're setting the YEAR et al. variables in the
PROMPT_COMMAND variable.
If that's the case, it's *less* wrong, but only a little bit. You still
have the issue that the date might change while you're sitting at a
stale shell prompt from yesterday, with stale date/time variables.

Actually, that is what I do want (the time—the date not so much).
That tells you when the last command finished. Press Return before
you start a command and you get some idea of how long it takes
to run. For the current time, press Return.
I think putting the time±date in one's prompt is pretty popular,

Sure. That's a completely different problem from what Keith is describing,
though.

Putting date/time expansions into your PS1 (prompt) causes the system
time to be retrieved when the prompt is displayed. The date/time strings
are rendered and displayed, and then not remembered by the shell.

What Keith is doing is storing components of the date/time in variables
at shell startup, and then using them at some unspecified point in
the future to create a directory. This causes issues when the shell
survives through a day transition (midnight).

Keith's workflow might work well if he rigorously logs out (or at least
closes all shells) every night before midnight, and logs in (or re-opens
his shells) in the morning. But even then, it's still a questionable
implementation. Anything that relies on a human being to remember to
do something is eventually going to fail.

A safer implementation would retrieve the current date at the moment
when it's needed. Your PS1 string does this.

[David Wright]

Post by Greg Wooledge

Post by David Wright

Post by Greg Wooledge

Post by Keith Bainbridge
The day# in my command prompt increments when I start in the morning. Maybe I need to press enter.

That makes it sound like you're setting the YEAR et al. variables in the
PROMPT_COMMAND variable.
If that's the case, it's *less* wrong, but only a little bit. You still
have the issue that the date might change while you're sitting at a
stale shell prompt from yesterday, with stale date/time variables.

Actually, that is what I do want (the time—the date not so much).
That tells you when the last command finished. Press Return before
you start a command and you get some idea of how long it takes
to run. For the current time, press Return.
I think putting the time±date in one's prompt is pretty popular,

Sure. That's a completely different problem from what Keith is describing,
though.
Putting date/time expansions into your PS1 (prompt) causes the system
time to be retrieved when the prompt is displayed. The date/time strings
are rendered and displayed, and then not remembered by the shell.

Yes, I think I misinterpreted what you were criticising, sorry.

Post by Greg Wooledge
What Keith is doing is storing components of the date/time in variables
at shell startup, and then using them at some unspecified point in
the future to create a directory. This causes issues when the shell
survives through a day transition (midnight).
Keith's workflow might work well if he rigorously logs out (or at least
closes all shells) every night before midnight, and logs in (or re-opens
his shells) in the morning. But even then, it's still a questionable
implementation. Anything that relies on a human being to remember to
do something is eventually going to fail.
A safer implementation would retrieve the current date at the moment
when it's needed. Your PS1 string does this.

Yes, he should be able to achieve his ends with no variables
at all (using bash), unless he wants to parameterise the
/mnt/data/keith/Documents/ part of the path. Something like:

#!/bin/bash
mkdir -p "/mnt/data/keith/Documents/copying-$(date --iso-8601)-dir" && cd "$_"
cp whatever-files-are-to-be-copied ./

If that file is executable and called /path-to/copy-files,
then an entry like:

*/2 * * * * [ -x /path-to/copy-files ] && /path-to/copy-files

in his crontab should run it every couple of minutes (to test it).

Using --iso-8601, or a format like it, will make the directories
sort in the right order. Was using daynumber an attempt to skirt
round this issue? If so, the year had better come first.

Is this what the OP is after?

Cheers,
David.

[Keith Bainbridge]

Post by Keith Bainbridge

Post by Greg Wooledge

Post by Keith Bainbridge
From the tab I had used earlier, ran source .bashrc
then
:/tmp/205.2024 $> mkcd /tmp/day$DOYR.$YEAR

So you're setting those variables one time inside your .bashrc file?

The variables are in .bashrc. I believed that was a good place to keep them. I'll check the original tomorrow, but it looks like a couple are there by default

I was wrong here. There is no sign of variables my original .bashrc.  I
don't know where I got that idea from.

Post by Keith Bainbridge
I intended to list all variables in my original note. Seems I need to put them else where

Post by Greg Wooledge
This is quite bad. What happens when you have a terminal window that
you open before midnight, and then continue using *after* midnight?
Those variables will still be set to the previous day's values.

The day# in my command prompt increments when I start in the morning. Maybe I need to press enter. I'll check tomorrow and confirm

So when I opened my xterm this morning, I saw:
 ***@lenv0

 Tue ***@19:19:30 205.2024 AEST
 :~   $>

You have new mail in /var/spool/mail/keith


Pressed enter, and the day# updated:

 ***@lenv0

 Wed ***@09:48:36 206.2024 AEST
 :~   $>

Post by Keith Bainbridge

Post by Greg Wooledge
Also, why did you start three separate threads for this question?

I wasn't aware I had started new threads. I realised as soon as I read that comment, that all I had to do was reply to group with the addenda

Post by Greg Wooledge

Post by Keith Bainbridge
So, now question is how can I get cron to 'source .bashrc' I'm thinking
add that line to my script?

I would advise against it. Who knows what kind of silly things your
.bashrc may do that you wouldn't want an automated script to pick up.
Even if you verify that your .bashrc is "cron safe" right now, you
might add something to it a year from now, forgetting that you need
it to remain "cron safe", and accidentally break the script.
Any command that's supposed to act based on the current day will need
to perform its *own* request to fetch the current day from the system
clock. It cannot rely on an environment variable set some unknown
length of time in the past.
Use the date command or bash's printf %()T inside your bash script to
get the current date.

Rightio

Post by Greg Wooledge
Moreover, never make *two* calls to the system clock in the same script.
If you need the Julian day number (%j) and also the calendar
# WRONG
julian=$(date +%j)
dom=$(date +%d)
That calls out to the system clock twice, and gets two different values.
If the script is executed right at midnight, you might get two different
days.
Instead, only make *one* call. There are a couple different ways to do
this. One is to get all the date(1) fields you need at once, and then
read -r julian dom < <(date "+%j %d")
Another is to fetch the epoch time value (%s) and then use that value
printf -v now '%(%s)T' -1
printf -v julian '%(%j)T' "$now"
printf -v dom '%(%d)T' "$now"
(Bash 5.0 added an EPOCHSECONDS variable as well.)
This same concept applies in any programming language you use. It's not
just a shell script thing, even though I'm showing shell script examples
here.

Greg, I've tried those suggestions in xterm and it looks promising.  
I'm guessing I should add the printf statements to the beginning of my
script?


I also see some other contributions to the thread. Thanks ALL

More bed time reading for later tonight.


It has taken 10:00 to get done what I can today - 9 hours all up.
--
All the best

Keith Bainbridge

***@gmail.com
***@gmail.com
[phone]

UTC + 10:00

[Greg Wooledge]

Post by Keith Bainbridge
 :~   $>
You have new mail in /var/spool/mail/keith
 :~   $>

Please tell us *exactly* what "opened my xterm this morning" means.
Did you run a new instance of xterm? Did you un-minimize an existing
instance?

Are you using a persistent tmux or screen session?

Here are a few commands whose output might be helpful:

declare -p PS0 PS1 PROMPT_COMMAND
ps -fp $PPID
trap DEBUG
PS4='+ $BASH_SOURCE:$FUNCNAME:$LINENO:' bash -ixc : 2>&1 | grep YEAR
PS4='+ $BASH_SOURCE:$FUNCNAME:$LINENO:' bash -ilxc : 2>&1 | grep YEAR

[Keith Bainbridge]

Post by Greg Wooledge
Another is to fetch the epoch time value (%s) and then use that value
now=$(date +%s)

Good evening All - especially Greg

This process has worked as required for the past week.

Thanks Greg for the very meaningful suggestion.
--
All the best

Keith Bainbridge

***@gmail.com
***@gmail.com
[phone]

UTC + 10:00