Audioscrape: Building in Rust When Everyone Said I Shouldn't

67 pointsposted 7 hours ago
by lukaesch

Item id: 41766551

36 Comments

lyang2821

3 hours ago

I bootstrapped Lapdev (https://lap.dev/) all in Rust too. Axum for the backend and Leptos for the frontend. Sharing structs between backend and frontend is quite nice.

Besides the memory safety, type system etc, Rust gives you the confidence that you know it can scale in pretty much all scenarios in terms of performance.

lukaesch

an hour ago

Congrats on the impressive project! Sharing structs between the frontend and backend sounds very helpful. I've experienced the advantage of shared FE and BE code in JS/TS projects, and it’s definitely valuable.

What led you to choose Leptos for the frontend besides the shared code advantage?

bsnnkv

4 hours ago

I bootstrapped Notado[1][2] and Kullish[3] in Rust. Best tech stack decisions I ever made.

Similar setups (Rocket and Actix instead of Axum, Tera instead of Askama, Diesel instead of SQLx) and overall experience (incredibly cheap, reliable, and since these projects have been running a long time, amazingly easy to return to the codebases after months/years and make changes with a high level of confidence).

I distinctly remember how happy I felt the first time I realized that I had enough paying subscribers to offset the cost of the running the service!

[1]: https://notado.app

[2]: it had a decent amount of HN discussion last week - https://news.ycombinator.com/item?id=41697228

[3]: https://kulli.sh

lukaesch

an hour ago

That's awesome! Reaching the point where subscribers cover costs is a huge milestone. Rust's reliability making it easy to revisit projects is a big plus. How do you find Rocket and Actix compared to Axum? Why Tera instead of Askama? Would love to hear your take!

bsnnkv

14 minutes ago

Rocket/Actix/Axum are all similar enough; I've used them all and I find Rocket the most ergonomic for web apps, Actix for pure backend APIs. I have spent less time working in Axum since it is the newest, but I don't have anything "bad" to say about it.

I started with Rocket for Notado, but because there was a long period of pre v0.5 stagnation, I went with Actix for Kullish. These days I'd be happy starting a new project with either.

As for Tera over Askama, I don't think Askama was around when I started building Notado; Tera was the first templating engine I used in Rust and I learned its ins and outs pretty well, so now it's just the default thing that I reach for whenever I'm building a web app.

jamil7

2 hours ago

Awesome work. Yes, I’ve been interested in Rust for a while and have been hacking on a side project. It’s an API service in Rust, but for the CRUD management part I used Elixir/Phoenix and Phoenix gen auth to have something up quickly that’s not dependent on some third-party service.

lukaesch

an hour ago

Thanks. Interesting. What are some best practices for integrating Rust with Elixir/Phoenix in a project, especially when managing CRUD operations and maintaining system efficiency you learned along your side project?

aeturnum

3 hours ago

Neat!

Just fyi I tried to add Criminal[1] and it seems like it was...partially successful[2]? The "Enter RSS URL" page loaded for a while and then stopped.

[1] https://thisiscriminal.com/episodes

[2] https://www.audioscrape.com/podcast/criminal

lukaesch

3 hours ago

Thanks for the hint!

We hit the front page on Hacker News and received many submissions, which led to network errors when fetching some podcasts. I've just added retry logic and scheduling for new podcast fetches and pushed these changes to production.

The Criminal[1] podcast is now added. Due to the current demand, our transcription queue is a bit backed up, so transcription for this podcast will appear in a few days. Please be invited to have a look the next days again.

I'm actively working on securing more GPUs to help scale this process efficiently so in the future you don't have to wait that long.

Would it be interesting if I add e-mail "transcription ready" notifications for those who submit a new podcast?

[1] https://www.audioscrape.com/podcast/criminal

aeturnum

2 hours ago

Oh yah I totally understand the current circumstances, just trying to be helpful in debugging.

I think a notification feature makes a lot of sense, but it depends on how long the wait tends to be. In terms of setting expectations it might be better to display the current backlog and an estimate about when the transcription might be done (though of course both would be even better than either).

lukaesch

an hour ago

Thanks for the feedback! I'm glad to hear you find the notification idea useful. I'm considering displaying the current backlog and estimated completion times as you propose.

How would you expect to get an understanding of the backlog? Would a dedicated page for the entire backlog be helpful, or would you prefer to see which episodes are being transcribed on each podcast page?

Any insights on what would be most helpful for you would be greatly appreciated!

erazor42

4 hours ago

> Has anyone else bootstrapped using Rust? What were your experiences?

Yes, we are building a parsing platform in rust. (~50k loc) Stack: actix + diesel (async) for PG.

Getting the right project structure was not simple (that could deserve a blog post). Performance is super nice if you do heavy IO.

Beware of memory usage handling with actix.

Day to day dev is a joy, once it compile you can be confident it will work as expected.

Never .unwrap / panic !

Use clippy to check your code.

Once you have your project structure done, adding a new web handler is as fast as adding one in Django or Flask.

lukaesch

4 hours ago

I can confirm this! Initially tried shortcuts with unwrap, but once the structure is solid, it's on par with TS/Python frameworks.

fHr

5 hours ago

I actually think Rust is amazing I recently discoevered some rewrites in web developement in rust and they made build times on windows corpo bloat machine way way faster than conventional js/ts implementations so I would say go for it and all power to you, also many are just salty because it's another new thing and new/change = bad

lukaesch

an hour ago

Thanks for the motivation! I completely agree - using Rust to build a web app is truly refreshing. I've worked with Java, TypeScript, and Python in recent years, and it's gratifying to catch most errors at compile time instead of during runtime

samier-trellis

7 hours ago

> Learning Investment: Deepening expertise in a language with growing demand, potentially opening future opportunities.

If it's just you, then that reason alone justifies it.

lukaesch

6 hours ago

Yes, it's just me right now. I'm moonlighting after my day job and bringing my kid to bed.

lispisok

5 hours ago

How did you get your first 500 users?

lukaesch

5 hours ago

Just posting on X (replying under trending posts or in 'build in public' communities).

Some people used the share feature of Audioscrape to emphasize their points in discussions on X. For example, 'Here, this is what Zuckerberg said recently about Y <Link to moment>'

echelon

5 hours ago

Love your story here! You're not doing it wrong at all.

> Has anyone else bootstrapped using Rust? What were your experiences?

We scaled several websites to 6M+ MAUs on Actix/SQLx and ran that on the lowest tier VMs.

Rust is God Mode for efficient and scalable HTTP services. Moreover, the type safety and robust type system give us incredible confidence that our software is defect free.

lukaesch

5 hours ago

Thanks for sharing your experience! It's impressive to hear about your success with Actix/SQLx and Rust's scalability and efficiency. Knowing Rust can handle such scale is encouraging and gives us confidence for future growth!

While we currently have fewer users, I've also found Rust's type safety and robust system invaluable for building reliable software. In my experience, if it compiles, it just works—quite a contrast to dealing with runtime error minefields in Python, Java, or TypeScript.

katangafor

4 hours ago

That is absolutely wild!!

I've put off learning rust for so long. I really tried to get into it once, and ended up learning clojure instead lol. Rust just had too many concepts at once for me to learn, probably because I've only ever used garbage-collected languages.

But hearing this type of stuff always makes me want to try again

lukaesch

an hour ago

Absolutely, give it another try! I've spent much of my life with GCed languages too. Axum, SQLite with SQLx, and Askama form a dev-friendly combo.

Once you grasp Rust's concepts, you'll find the control and efficiency rewarding. It's worth the effort!

peterpans01

5 hours ago

"Database: SQLite with SQLx for type-safe queries"

I might be ignorance. But is it not safe? How do you back up your data?

lukaesch

5 hours ago

Thanks for asking! Using SQLite with SQLx in Rust provides type-safe SQL queries, which enhances safety by catching errors at compile time.

As for backups, the setup is straightforward: just periodically copy the SQLite file to a secure location. Since it's a single binary and a database file, this keeps things simple and low-maintenance.

pornel

4 hours ago

I hope you're calling the `backup` command in sqlite. A simple copy can leave sqlite db files in an inconsistent state from which sqlite can't recover.

lukaesch

an hour ago

Thanks for the hint. I didn't know that. So far nothing happened, but I will start using the backup command.

bravura

5 hours ago

How is transcription done?

lukaesch

5 hours ago

whisper + pyannote + custom LLM pipeline to reduce errors, extract and attach speaker labels

BetterWhisper

an hour ago

Are you running whisper on that same $7 Server?

miki123211

2 hours ago

How do you interface with the Python ML stack from Rust?

lukaesch

an hour ago

Whisper and Pyannote via Replicate to spin up a container for transcription and diarization. It returns word-level timestamps and speaker turns.

The LLM pipeline is built into the Rust code and makes calls to the OpenAI API.

aleksjess

5 hours ago

4k loc in a single file must be brutal

lukaesch

4 hours ago

Thanks! I was inspired by levelsio's meme about having MVPs in a single index.php file. Traditionally, I've organized codebases into folders, but I started questioning its necessity.

Folders often just add an extra layer to search through. It's basically a search param. With Neovim and strict naming conventions, I've found managing everything in one file works quite well. Keyboard navigation can make folders feel like a hassle in the vi context. This setup has been effective so far, though potential downsides might appear later on.

samatman

3 hours ago

Might want to check out Harpoon: https://github.com/ThePrimeagen/harpoon/tree/harpoon2

No comment on your one-file approach one way or the other, but like all of us, you will need to deal with tree-shaped projects, I've found Harpoon to be a good solution for this. Global marks can only get you so far.

Good luck with Audioscape btw.

diggan

5 hours ago

Can be, can also be just like using multiple files, especially since OP seems to be using neovim. Open a buffer per area and work away, just like using tabs/panes with multiple files.

Personally I wouldn't do it either.