cmacleod4
9 months ago
The first major release in 27 years. 64-bit internal structures, so data can be huge. Full unicode with all the funky new emojis. Zip filesystems, etc., etc.
There's lots of new stuff, and some old cruft has been dumped, so some programs may need a few updates, but there's still a high level of compatibility. The page above links to release notes with details of what's in and what's out.
wduquette
9 months ago
The Zip filesystem stuff is wonderful change to see: it takes a number of techniques that were common in the community (if you had the right tools and knew how to use them) for building standalone applications, and makes them part of the basic toolkit in a standard way. It's a truly excellent change, and I'm glad to see it.
packetlost
9 months ago
Can you explain what zip filesystem is?
mdaniel
9 months ago
A handler for treating .zip (and .jar and .tar ...) files as if they were a volume mounted at /home/user/foo.zip - e.g. https://github.com/openjdk/jdk/blob/jdk-21-ga/src/java.base/...
It may be easier to reason about when thinking of the way $(mount -o loop) works with .iso files -- a file that is a container for other files that one can mount as if it were a filesystem
I was expecting pathlib in Python <https://docs.python.org/3/library/pathlib.html> to have one since a lot of Python distributions ship the standard library in .zip files but evidently not. Python gonna Python in that way
Firefox actually used to ship with that same "jar:" protocol handler, too, and I made good use of it for reading the javadoc html which was shipped inside zip files and was indescribably easier than trying to manage all the .html files in a Java 8 SDK distribution. They made heavy use of this because a lot of their internals were in .xpi formats (which is also a .zip file) but they recently dropped it because Firefox gonna Firefox^W double down on some random shit other than making a sane browser
formerly_proven
9 months ago
Python has something kinda like VFS support just as a very special case for importing stuff and reading associated data files, see Anchor ("mount point") and Traversable ("virtual Path") in https://docs.python.org/3.12/library/importlib.resources.abc... - the actual implementation is done in finders et al (https://docs.python.org/3.12/library/importlib.html#module-i...)
eadmund
9 months ago
Emacs implements something like this too! It’s pretty useful to be able to just enter a tarball and work with the files in it.
It looks like https://github.com/cybernoid/archivemount provides a similar experience in the shell, although one would have to mount the file first.
boltzmann-brain
9 months ago
they should do the same but for git repositories too
kragen
9 months ago
not .tar
sbstp
9 months ago
I'm not a TCL user, but from the description of wduquette is sounds similar to Python's zipapp
oldlaptop
9 months ago
The definition of "standalone application" is a bit stronger - what's always been possible with "Tclkits" has been to embed the application code in an interpreter binary and distribute that, and the new core zipfs supports that as well.
wduquette
9 months ago
Yes, exactly. You can package up your entire scripted application, plus all resources, into a single executable, and distribute that single file.
Zuider
9 months ago
Why did they remove tilde '~' as a convenient shortcut for the Home directory?
bch
9 months ago
Tcl Improvement Proposal (TIP) 602[0].
throw0101b
9 months ago
One example from the document:
> Consider the naive attempt to clean out the /tmp directory.
> cd /tmp
> foreach f [glob *] {file delete -force $f}
> A file ~ or ~user maliciously placed in /tmp will have rather unfortunate consequences.
em-bee
9 months ago
i once managed to create a directory named ~ using the mirror tool written in perl. then i naively tried to remove it using "rm -r ~" and started wondering why removing an empty directory would take so long, until it dawned on me...
i learned a few new habits since then. i almost never use rm -r and i avoid "*" as a glob by itself. instead i always try to qualify "*" with a path, remove files first: "rm dir/*"; and then remove the empty directory. "rmdir dir/"
if i do want to use rm -r, it is with a long path. eg in order remove stuff in the current directory i may distinctly add a path: rm -r ../currentdir/*" instead of "rm -r *"
related, i also usually run "rm -i", but most importantly, i disable any alias that makes "rm -i" the default, because in order to override the -i you need to use -f, but "rm -i -f" i NOT the same thing as "rm". rm has three levels of safety: "rm -i; rm; rm -f". if "rm -i" is the default the "rm" level gets disabled, because "rm -i -f" is the same as "rm -f"
sfink
9 months ago
My main safety habit is to avoid slashless paths.
Bad:
rm *
rm ./*
rm /tmp/d/*
rm */deadmeat
rm d/*
mzs
9 months ago
rm -rf "$TSTDIR"/etc
sfink
9 months ago
Fair! Upvoted.
I guess I'm not likely to type that into the shell, or if I do, I then tab-complete to expand it.
I could definitely see myself using that in a shell script, though. I tend to do validity checks there:
if ! [ -d "$TSTDIR" ]; echo "$TSTDIR not found, stupid" >&2; exit 1; fi
if [ -d "$TESTDIR" ]; then
rm -rf "$TSTDIR"/etc
fioneshtein
9 months ago
In bash, `set -u` or `"${TSTDIR:?Error: TSTSDIR is required.}/etc"` protects from such errors.
niobe
9 months ago
My safety technique is to echo the commands before I do the actual commands as a sanity check, e.g.
for i in $(find something); do echo "rm -f $i"; done
(bash example as my TCL is rusty)
dundarious
9 months ago
Change your do block to `printf %q\ rm -f "$i" ; echo` and it won't lie about spaces. In case HN has "trimmed" my post in some way, as it often does, that's: percent q backslash space space. Works in bash/zsh, but not dash, probably not whatever your sh is. Can make a function of it trivially, but you have to handle the $# -eq 0 case, return whatever printf returns, etc.
somat
9 months ago
When deleting, if it is more than a few specifically named files I will use a "find ... -delete" invocation.
I like it for two reasons. Find feels like it has more solidly defined patterns and recursion than shell globing and by leaving off the "-delete" it give me a chance to inspect the results before committing to my actions.
kstrauser
9 months ago
Without testing, I wonder if find follows symlinks. I’m pretty sure rm doesn’t.
Edit: Just checked and find doesn’t by default.
progmetaldev
9 months ago
Very cool of you to post this. Too many people won't post stories like this, but I've done very similar multiple times. I think it definitely helps reinforce proper habits, and is the best way to cut your teeth on technology. It's also great for anyone new to read something like this, and be able to avoid something so devastating, and maybe make lesser mistakes, but still learn from both!
em-bee
9 months ago
when you get to be as old as i am, these are the war stories you share with your kids and grandkids around the campfire ;-)
like the one from my colleague who once fat fingered fsck into mkfs and i lost my personal homepage because of it. what makes me uncomfortable about that story is that it was not my fault. if it were it would have been easier to tell. but at the time i was quite frustrated and my colleague felt that despite me trying to not get angry at him. i still feel really bad about my reaction then, adding to his predicament, since he had to live with the guilt about losing our website and everyone's personal home directory. it's bad when a mistake causes you to loose something personal, but so much worse when you loose someone elses stuff.
talking about mistakes is how we learn from them. the important part is not to get embarrassed about them. however that requires an environment where we are not blaming each other when something goes wrong.
i could have made that mistake myself. and i applied this lesson to my own learning as if i had.
blessed be the pessimist, for he hath made backups...
progmetaldev
9 months ago
Absolutely, the worst mistakes are the biggest opportunities for learning. Destroying your own files is painful. Destroying EVERYONE'S files is a lesson that is more painful and something you will be more careful not to repeat.
I try to tell as many of these stories in person as possible, to let everyone know that unless you have dealt with this kind of accident, you're either in the 1% or due for your turn. I'd like to think that sharing these stories might at least give someone some pause before they go ahead and throw caution to the wind.
barosl
9 months ago
> if "rm -i" is the default the "rm" level gets disabled, because "rm -i -f" is the same as "rm -f"
You can use "\rm" to invoke the non-aliased version of the command. I made "rm -i" the default using an alias and occasionally use "\rm" to get the decreased safety level you described. I think it is more convenient that way.
sweeter
9 months ago
I love zsh auto completion for this stuff. It automatically escapes really messed up paths like paths with new lines or emojis and crazy characters like that. Its really rare but I still intentionally practiced removing these things just so I can do it safely if it ever happens.
kristopolous
9 months ago
I've long fantasized about a tool I call "expect" that safeguards against crazy stuff like that.
It has a syntax of your expectations, functionally existing as a set of boundaries, and you can hook it to always run as a wrapper for some set of commands. It essentially stages the wrapped command and if none of the boundaries are violated it goes through. Otherwise it yells at you and you need to manually override it.
For instance, pretend I'm ok with mv being able to clobber except in some special directory, let's call it .bitcoin or whatever. (chattr can also solve this, it's just an example). The tool can be implemented relying on things like bpf or preload
Originally I wanted it as a SQL directive ... a way to safeguard a query against doing like `update table set field=value expect rows=1` where you meant to put in the where clause but instead blew away an entire column. I think this would be especially useful surfacing it in frameworks and ORMs some of which make these accidents a bit too easy.
progmetaldev
9 months ago
When it comes to SQL, I will often write a SELECT with very explicit search (WHERE) criteria for this very reason. Then copying that statement, commenting the original, and pasting to change into an UPDATE or DELETE statement seems to be a technique that works well for me. The SELECT tells me exactly what I'm going to UPDATE or DELETE, and once I have that, changing the syntax is very minimal. In the case of an ORM, you might have to write a tool that only listens on LOCALHOST to run these statements first.
kristopolous
9 months ago
I always write the where first. It's kinda like thinking in RPN or postfix. I put the parts in out of order in a way that prioritizes the minimization of error.
But this is stupid. These are computers, we can make whatever we want. Executing a delete or update should, if one desires, not have to be database knifeplay.
progmetaldev
9 months ago
I know what you mean, I do the same. I agree, but at the same time, it's difficult to start building in protections for the user. Where do you start and where do you stop? I have been forced to do the extreme to protect the user, and then you are asked why things are so difficult to use. I think to make something for someone that concentrates in the technology, as well as a beginner, means you've got to give up so much power (or create a secondary syntax/interface for both audiences). It would be nice to be able to set modes, but then it's going to be database specific unless it has proven itself to be useful across engines. Like most standardization, then you play syntax games between vendors. It would be nice to at least be able to write an UPDATE or DELETE statement with a leading character or keyword to display affected rows.
kristopolous
9 months ago
It's completely Optional safeguards. Add long as it's optional, I advocate for having as many of those as people can imagine
progmetaldev
9 months ago
I understand, but with how much of a change to the language? Such a change would take an enormous amount of time to make it into the ANSI/ISO SQL standard, and what database would start to implement it first, and which would hold out as long as possible?
I don't disagree that it's impossible, but how do you get the syntax standardized at this point? Do you get various dialects, or an agreement between vendors? Look how slowly the standard moves, when do we get this where it's usable in most popular RDBMS?
kristopolous
9 months ago
The venn diagram of query support between SQL vendors is much closer to a flower than you think.
Just implement it for one and if it works, the others will add it
progmetaldev
9 months ago
I have upvoted you for each comment you've made, but I feel like it's not that simple. Even just getting a single vendor to implement it is a huge undertaking. I know that you and I see the value in it, but I don't feel like we're the first to see that. There's a reason behind not implementing this feature, and it's the complexity that lies behind such a feature, like most things. This seems like one of those recursive and interactive features that don't fit into SQL. Does it present the rows that will be updated or deleted, and then ask if you wish to perform the operation? That doesn't work like anything SQL based, and I feel that's why we don't have it. I appreciate the back and forth on this, and am curious as to how you think it should be handled, if there's a way to fit in the way SQL works.
kristopolous
9 months ago
If the expectation is not met then it rolls back and fails. I implemented a slipshod version of it years ago for a previous employer (it got the job done with a lousier syntax)
Here's a list of 1,000 postgres extensions, it's not a big deal: https://gist.github.com/joelonsql/e5aa27f8cc9bd22b8999b7de8a...
Things are way more modular than they used to be.
I can probably do it again and just try to get attention for it.
specialist
9 months ago
From the hip, maybe something like "UPDATE DRYRUN ...". It'd report how many rows would be updated.
Or... "DRYRUN UPDATE ...", which is more like "EXPLAIN UPDATE..."
Thoughts?
progmetaldev
9 months ago
Sounds like a solid idea, but I feel like just replacing your UPDATE with a SELECT COUNT(whatever_column_is_indexed_from_your_where) would be a good practice. If your DBMS supports an external language, that might be the best idea, so you can write your own logic, while keeping mostly everything in the database itself.
I only mean this from a more ANSI SQL side of things, where you might want to build your skills up to use as much of the standard as possible, until it's no longer possible and make sense to dip into platform specifics. I used to build code around being cross-platform, but at the same time realized that it's more useful to learn the ANSI standard, and then break free with LOTS of useful comments where it makes sense to do things more efficiently and with safety that you don't get normally.
danielheath
9 months ago
For sql specifically, “limit 2” is my default way to write “expect 1”; if it affects two rows, I know that I have screwed up, whereas “limit 1” can be wrong without my noticing.
kristopolous
9 months ago
That's not a terrible solution although expect sounds like a simple safety mechanism for feeble-minded people like me who do simple queries.
I actually know postgres people. I should probably ask them
kragen
9 months ago
just to clarify, this has nothing to do with the "expect" that is the other major application of tcl other than tk?
kristopolous
9 months ago
None.
I was just reminded of a good idea I never implemented
x-shadowban
9 months ago
Oh yeah, re: SQL expect - I always wish joins had a "cardinality assertion", like a regex *?+ (or ! for exactly one)
rixed
9 months ago
You could also create a file named "-i" in your home dir.
cassepipe
9 months ago
Why not learn to enjoy life knowing you get a second chance instead ?:
em-bee
9 months ago
because then i become to rely on stuff being in the trash and i'd be less careful when deleting, which means i have to double check when i clean the trash. that's extra work. . and since the trash is one single folder for the whole desktop, that means the trash is full of stuff from all over the place, making a review extra hard. in most cases i know something needs to be gone, so i'd rather delete it on the spot.
besides that, the primary reason for deleting stuff is to gain space. moving things to trash doesn't help with that.
what i sometimes do though, when mass deleting, is to move stuff to be deleted into a new folder (usually called "del" or "delete"). then verify the contents of the folder before removing it.
what would be more useful is a kind of trash implementation that does not take space, in that it keeps files around but reports them as unused space that can be overwritten when space is needed. kind of like undelete is possible on some filesystems. so that gone is gone because i can't control when deleted space gets used up, but in a panic situation i can revert the most recent deletes.
amelius
9 months ago
Isn't that just tilde-expansion happening at the wrong moment?
AlienRobot
9 months ago
I once created a file named *. Sweats were sweated that day.
orthoxerox
9 months ago
I've heard that one of the Unix founding fathers had a directory with 125 files that all had single-byte names: one for each ASCII symbol except slash, dot and null. He would then test any new utility against this directory and chew the careless programmer out if it couldn't correctly handle every one of these names.
cmacleod4
9 months ago
In the long run, special cases like that often turn out to be more trouble than they are worth. If an ordinary file happened to start with '~' it would not be handled correctly. So you either accept or ignore that potential problem, or you have to write extra code to work around it. It's safer to not have such special cases at all.
mirekrusin
9 months ago
Should be starts with ~/
cmacleod4
9 months ago
No, ~abc will be interpreted as the home directory of user abc in Tcl 8.*
eviks
9 months ago
Then you fix this mistake, not remove ~ completely
Y_Y
9 months ago
It's not a mistake
mirekrusin
9 months ago
If it's not a mistake then why they took it off?
cmacleod4
9 months ago
The original design decision was to behave like a shell, where ~ means your own home dir and ~fred means fred's home dir. With the benefit of experience, this is now seen to be unwise and a different decision has been made.
eviks
9 months ago
So they copied the mistake from some shell (you don't need much experience to realize it's unwise not to use a separator to separate)
Koshkin
9 months ago
Just recently I used '~' in the remote target path in the scp command, and, somewhat unexpectedly, it created the directory with that name and put the files there.
isr
9 months ago
In 99% of cases, yes - this is a pain in the backside. Long ago I adopted something from the plan9 way of doing things (when I was heavily using acme).
Just symlink /h to /home. So ~user becomes /h/user, in places where ~ is not expanded for you.