Syncthing: Open-Source Continuous File Sync with Privacy in Mind

146 pointsposted 14 hours ago
by EchoForge
(github.com)

54 Comments

evanjrowley

14 hours ago

I had to stop using it after loads of conflict files piled up over the years in my notes folder.

I'm trying to switch to WebDAV for notes. Considering Caddy+WebDAV[0] or Peergos[1] or SeaweedFS[2], but not Next/OwnCloud.

Is still consider using Syncthing for other files types.

[0] https://whhone.com/posts/webdav-syncthing/

[1] https://github.com/search?q=repo%3APeergos%2FPeergos%20webda...

[2] https://github.com/seaweedfs/seaweedfs/blob/master/weed/comm...

soupbowl

14 hours ago

I had similar issues years ago and it was frustrating. What I do now is have syncthing on a 'server' that is always on and everything syncs to the server and not between each device. That reduced my conflicts by 99%. Another issue I had was accidentally deleting a folder and having it deleted off of all my devices.

To solve that issue I have my server do ZFS snapshots on my syncthing folder, which makes for easy recovery. With those 2 things being done, I can't say enough good things about syncthing. Compared to nextcloud it is almost maintenance free.

ta988

12 hours ago

Syncthing has staggered versioning that keeps several versions at different timepoints this may have solved that issue.

EGreg

14 hours ago

Why not just use rsync -a periodically over an encrypted ssh connection? It works across Mac/Linux etc. and is just as efficient, no?

If you really want to be fancy, you can deploy a script to watch files for changes and do rsync immediately when it happene.

aniviacat

13 hours ago

> If you really want to be fancy, you can deploy a script to watch files for changes and do rsync immediately when it happene.

I use a tool that does that, it's called Syncthing.

EGreg

13 hours ago

Yeah but if you’re just syncing everything to and from a server anyway, why bother with it?

Plus, rsync -a never deletes files.

vetinari

2 hours ago

Because you want syncthing to delete files more often than not. When you want the reverse, it's a single property on the shared folder (ignoreDelete), feel free to set it.

ta2234234242

8 hours ago

Plus why doesn't sync thing support rsync protocols anyway? It's not like it couldn't, right?

LeoPanthera

7 hours ago

They're not the same thing. rsync is unidirectional, and only between two endpoints. One side is always overwritten by the other side.

syncthing is bidirectional, syncing changes in either direction, and supports syncing any number of clients, not just two.

Watching and syncing immediately - yes, that's what syncthing does. Why reinvent it with something worse?

dmd

13 hours ago

You can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem.

EGreg

13 hours ago

Not sure why this would be easier or better

FTP is not encrypted. Unless you use SFTP maybe. (I wrote an SFTP and FTP-over-SSH client in Visual Basic like 20 yesrs ago lol).

And even then, you miss out on all the optimizations with prolly trees and other goodness that rsync has.

No, sorry, that solution sucks. CVS? That’s super outdated.

fallingsquirrel

13 hours ago

It's a reference to the infamous dropbox comment: https://news.ycombinator.com/item?id=9224

Anyways, if you're at the grocery store and you need to see your shopping list, and you're happy with your current workflow of opening up Termux and typing `rsync -avrsomgwtfbbq me@server/shopping-list.txt` with your thumbs, then by all means keep doing that! For the rest of us, there's Syncthing.

EGreg

13 hours ago

Yeah but on the substance, none of your countrexamples make sense

SyncThing doesnt actually have an iOS client that syncs reliably. I tried some, when I was test driving it and Resilio Sync and others.

https://forum.syncthing.net/t/syncthing-for-ios/16045

On your phone you’d probably just visit a website, and access your server via HTTP.

soupbowl

8 hours ago

I sync files across Windows, FreeBSD, Linux and Android. Your solution wouldn't work for me as easily as Syncthing.

LeoPanthera

14 hours ago

Conflict files are good. It means you changed the same file in multiple locations and Syncthing didn't overwrite or delete any of them.

Much like forest fires, only you can prevent conflict files.

KennyBlanken

14 hours ago

Nah. They're not alone. For me, syncthing would just randomly decide an updated file conflicted; I had two computers and a smartphone and was pretty careful about this stuff.

It was its only fault - but it was easier to figure out what the actual most-current file was.

Nextcloud does the same thing occasionally, and it's not intuitive at all trying to figure out which one is the proper one to keep.

ta988

12 hours ago

There was a bug in the android client a while back that was causing that it is much better now.

baby_souffle

14 hours ago

> I had to stop using it after loads of conflict files piled up over the years in my notes folder.

SyncThing is still still _loads_ better than Next/Own-Cloud unless you also need the extra baggage those two bring.

Having said that, "conflict-sync" files are my current issue.

I have several devices that sync to a local central device and a remote central device and I'm struggling to determine if the "conflicts" that are _not_ conflicts are because of this "dual-honed" approach or not.

Filligree

14 hours ago

I've had to reset it multiple times due to the same sort of issue. There doesn't seem to be a good way to resolve conflicts; at least, none that reliably works.

generalizations

14 hours ago

You have to ensure there's continuity - at least something online and connected at all times, to ensure that the system can know and have distributed the latest version of the files.

I just have it running on my desktop, and at this point, the system's been running for nearly 6 years, and the file conflicts are rare. They only happen when I e.g. do edits on my laptop offline, then go home and do more edits on my desktop before letting my laptop sync.

zelphirkalt

14 hours ago

Since I started using Syncthing, I have found many cases of "just make a new folder and share it with selection of other devices" and now, that I am running it on a server, it is also always online, ready to share with any device that comes online. It is just so neat and handy!

aborsy

13 hours ago

Syncthing is probably the best sync software on market right now in my view. I have had very positive experience with it.

I have an always on device that runs various applications including syncthing. Works great and sync is fast. Like send a movie from a computer to phone, and it gets there fast.

I have started to use encrypted folders (beta feature). If you have a VPS, this might be a useful feature for you.

I like that the transfers are peer to peer, and syncthing has been very secure so far.

kristjank

14 hours ago

SyncThing has been the only piece of open-source software which I've fire-and-forget installed and it's working indefinitely.

edarchis

14 hours ago

I use it on so many platforms and even sometimes in three-way like my Obsidian vault that syncs between my laptop, my phone and my tablet. I barely ever had any conflict. Amazing software.

tetris11

13 hours ago

Yep, good software and the dev (nutomic) is the main Lemmy coder, meaning the fediverse is in good hands.

bdjsiqoocwk

14 hours ago

Seconding the sentiment <3

yownie

14 hours ago

I've had the intention of testing it out for YEARS and finally got around to it last week or so. Really pleasantly surprised how well the NAT punching / relay servers handle keeping machines synced.

I wish I had tried it all those years ago. I wish excluding directories was a little more intuitive and there was a good way to administer config's on a machine not local to me. Maybe over SSH, I'm still learning.

danparsonson

13 hours ago

> ...a good way to administer config's on a machine not local to me.

If you can't access the web interface, is Wireguard an option? That's how I do mine.

naming_the_user

12 hours ago

You can use SSH forwarding.

ssh -L 12345:localhost:8384 remoteserver

Keep the ssh session open, maybe “watch date” or something

Go to localhost:12345 in your browser

Bosh

icelancer

14 hours ago

This system works extremely well unless you have a LOT of small files (like 200-400k or more). Then, unfortunately, it stalls/breaks in very weird ways.

We had to move on and go with more manual synchronization methods like using robocopy and so forth.

It's unfortunate given how awesome SyncThing + SyncTrazor works on Windows Servers. But I cannot recommend it highly enough if you don't have a system with a huge number of individual files - it works very well.

bitbasher

14 hours ago

Anecdotal data, but it feels like a majority of the nice software I look up on Github ends up being written in Go.

candiddevmike

13 hours ago

Go made static binaries popular, and those solve a ton of distribution challenges.

lmaoguy

14 hours ago

I found that it struggled with larger data sets. I have a directory that is 70TB and about 300,000+ files and unfortunately, it just can't handle that. Resilio (formerly BitTorrent Sync) handles it nicely and they just opened up to allow home users better access to it.

vibrant_mclean

13 hours ago

syncthing automtically connects with bunch of external servers automatically. Relays/discovery servers/stun etc. You can disable but they are enabled by default. They connect on first run and before you get a chance to disable - atleast on android. Everything may be encrypted but I still prefer to not connect to external servers

candiddevmike

13 hours ago

Yes, IMO the external connections and relays should be disabled by default/opt in for privacy reasons.

eightys3v3n

14 hours ago

I would like to reflect on all the positive sentiments here. I have been using it for years and I rarely have to look at or think about it more than once every two months. Even when I do need to look at it, it's just to realize I filled some drive and that's why it's not synchronizing anymore.

I have no issues day-to-day and it's relatively easy to set up. I haven't played with the zero trust encrypted folder syncing yet but I will eventually :p

I'm running it on Linux and Android but I started out with it on Windows. Thanks for the reminder about this software. I don't think I've donated in a couple years now.

kookamamie

14 hours ago

Whenever I need to share files between two Windows computers in a LAN, I'll use this tool.

It sounds kind of ridiculous, but just goes to show how crappy/difficult the Windows native file sharing is.

eternityforest

3 hours ago

Localsend is excellent for all the use cases SyncThing doesn't cover well

alok-g

9 hours ago

I recently set it up for syncing files between Windows and Mac. That results in many sync errors because of differences in allowed special characters in the file names. Anyone having recommendations for avoiding that? Thanks.

darkteflon

13 hours ago

Huge fan. Been using it for years with very few issues across Win10, MacOS, iOS, Ubuntu / PopOS, Steam Deck and even Quest 3. Fire and forget. Like Tailscale, don’t know what I’d do without it.

diego_moita

14 hours ago

Syncthing + Raspberry Pi + Wireguard + Gitea = There's no "cloud"! It is just someone else's computer.

danparsonson

12 hours ago

...+ Nextcloud + Joplin + Jellyfin + ntfy + Lychee + ....

I'm hooked on this stuff; the amount of high quality free software available now it's just incredible!

dang

14 hours ago

Related. Others?

Syncthing – A decentralized continuous file synchronization program - https://news.ycombinator.com/item?id=41281152 - Aug 2024 (23 comments)

Syncthing: A continuous file synchronization program - https://news.ycombinator.com/item?id=35879039 - May 2023 (124 comments)

Syncthing Anywhere with Tailscale - https://news.ycombinator.com/item?id=32375662 - Aug 2022 (42 comments)

Syncthing Stats - https://news.ycombinator.com/item?id=32012712 - July 2022 (4 comments)

Syncthing: The Data Deduplication Master - https://news.ycombinator.com/item?id=31278867 - May 2022 (2 comments)

Syncthing – a continuous file synchronization program - https://news.ycombinator.com/item?id=28859521 - Oct 2021 (230 comments)

Syncthing – continuous file synchronization program - https://news.ycombinator.com/item?id=28371802 - Aug 2021 (1 comment)

Syncthing: Syncing All the Things - https://news.ycombinator.com/item?id=27929194 - July 2021 (172 comments)

Open Source Continuous File Synchronization - https://news.ycombinator.com/item?id=27149002 - May 2021 (146 comments)

Syncthing is everything I used to love about computers - https://news.ycombinator.com/item?id=23537243 - June 2020 (159 comments)

Do not use Syncthing (2019) - https://news.ycombinator.com/item?id=23116462 - May 2020 (1 comment)

Emacs' Org-Mode and Syncthing = Perfect (2017) - https://news.ycombinator.com/item?id=23058358 - May 2020 (77 comments)

Syncthing: An open source Dropbox replacement - https://news.ycombinator.com/item?id=20466469 - July 2019 (41 comments)

Syncthing graduation day - https://news.ycombinator.com/item?id=18832517 - Jan 2019 (114 comments)

Syncthing Usage Data - https://news.ycombinator.com/item?id=13856552 - March 2017 (119 comments)

Syncthing - https://news.ycombinator.com/item?id=10331031 - Oct 2015 (1 comment)

BitTorrent backups – BTsync and Syncthing - https://news.ycombinator.com/item?id=10207935 - Sept 2015 (1 comment)

Syncthing: Open Source Dropbox and BitTorrent Sync Replacement - https://news.ycombinator.com/item?id=7734114 - May 2014 (184 comments)

scotty79

12 hours ago

I had a small issue with Syncthing-Fork on Android. The app defaults to having silent notifications so when I shared a new folder from other computer I didn't noticed it on my Android device and I thought it wasn't working because notifications are the only way you can accept and configure new share. Nothing else shows up even though I had the app opened all the time I was doing that.

em-bee

7 hours ago

you can open the web-gui from the menu and the notification should show up there

KennyBlanken

14 hours ago

Syncthing is a wonderful open source project that I thoroughly enjoyed using when my mobile device was on Android. Everything I needed was synced, all the time. Great performance, too. And it was really easy to set up syncing of a particular folder with random people for various projects.

I switched to an iPhone for greater privacy, and was crushed to find Syncthing doesn't support iOS.

I see that 5-6 years later they still don't. Sad, given the iPhone has a majority market share in the US (60%).

I use Nextcloud and it's....okay. There's some weirdness where files I've specified should be always available somehow aren't, but luckily the keepass client keeps an internal/backup copy and is smart enough to open that.

iCloud is just...a rolling dumpster fire. And easily the most frustrating thing about it is that there's no way to set up an iPhone without it immediately trying to sync everything to iCloud, like say, your iMessage encryption keys...

flanked-evergl

14 hours ago

I switched back to android for syncthing, totally worth it. I am on a galaxy s24, the battery life is not great compared to my last iPhone, and the camera takes poor photos, but other than that everything is better.

citiguy

14 hours ago

On IOS there's Möbius Sync, which works perfectly with Syncthing.

NotPractical

12 hours ago

You probably should've switched to GrapheneOS instead.

talldayo

12 hours ago

> I see that 5-6 years later they still don't. Sad, given the iPhone has a majority market share in the US (60%).

What goes around, comes around. Apple knows they make it hard on developers that want to make the iPhone behave like a normal computer. Same as any Apple platform, you pay the price premium for compatible software and not the other way around. Magic solutions don't manifest on developer-hostile platforms like that, and Apple makes sure of it. After all, they profit off every solution sold on their problem marketplace.

> there's no way to set up an iPhone without it immediately trying to sync everything to iCloud, like say, your iMessage encryption keys...

So much for "greater privacy" then, huh? Or... do we still think this is a negligible fact?