I think a large benefit to "DB in browser" is about the whole "local-first" software movement. A good overview is written here https://sqlsync.dev/posts/stop-building-databases/

The essence of the approach is that a large majority of FE apps have constructed quite complex caching layers to improve performance over "querying backend data" - take a look at things like Next.js or React Query <- as the post above mentions, they're essentially rebuilding databases. So instead this approach just moves the db to the browser, along with a powerful syncing layer.

I think it's an approach that deserves more attention, especially to improve DX where we end up writing a whole lot of database-related logic on clients. Mind as well then just use a database on the client as well

To the other good points, I'd add that when people start making queries that aren't O(1) or O(n), modern computers and networks are capable of reasonably conveniently moving around amounts of data where it becomes practical for remote client to use its rather substantial power to answer questions on a gigabyte or two of data with its own computing power, and if enough people are doing this at once the combined client power they can have can easily overpower any reasonable amount of cloud compute you might be willing to deploy for this problem as they run non-trivial queries. I may be happy for dozens of clients to download some data and chew through some O(n log n) with mixed O(n^2) components on their own whereas I would not care to provision enough cloud compute to handle it all.

I think people forget that as cheap as cloud compute is, client compute is even cheaper. Generally it's already paid for. (Marginal electricity costs are lost in the noise of everything else you're supporting a client user with.) And while the years of doubling every 1.5 years may be gone, clients do continue to speed up, and they are especially speeding up in ways that databases can take advantage of (more cores, more RAM, more CPU cache). Moving compute to clients can be a valuable thing in some circumstances on its own.

> If the data "belongs" to the server, why not send the query to the server and run it there?

We could, but if the data size is not that huge, sending it once to the client and then letting the client perform the queries can be desirable. The tool works without internet access, the latent is much better, all results are coherent etc

> If the data "belongs" on the client, why have it in database form, particularly a "data-lake" structured db, at all?

Just because it fits in memory doesn't mean that the shape of the data does not matter. A data structure optimised for whatever query/analysis you want to perform has a significant impact on how fast and efficiently you can perform those operations

This argument boils down to:

- database engines should always run on a server, never on the user’s own machine.

- local data is never larger than the database engine, and it’s better in all cases to move all of the local data to the server where the database engine runs.

- local data can always be moved to another machine, regardless of sensitivity.

Buy are these always true, all the time? It seems to me that there are many use cases (e.g. training local AI models) where these rules should not be so absolute.

There is an increasing subset of people (think those that used to work in MS Access, Excel power users) who learned SQL in a business IT course or on the job. They don’t have data sized to fit in a DB, but they do want to do analyses that use window functions and things that SQL makes more natural than Excel vtables or functions. They may have to give reports to an equally technical boss who would like to play with the report and explore assumptions using SQL.

The data size is typically not large; SQL and integrating with cloud-based spreadsheets is the selling point.

I built a web application that became a progressive web app because it needed to work offline because it was used by a pilot during their work, and being at high altitudes, internet is not guaranteed. We cached a bit on localStorage, and I'd loop trying to make HEAD requests (there was no "am I online" API available...) and once we knew you were online we would push the locally cached changes up to the DB. In the meantime we still displayed the current data as you've changed it as well. It worked out nicely.

There's a lot of static data that never changes, that if you can request once, and cache it somewhere, it becomes really useful. It's less requests coming to the server. Heck, Tumblr in the 2010s would display a lot of blog data on the dashboard such as follower count, likes count, etc they had outages all the time, when I started to see less and less outages was when they stopped showing all that blog metadata on the dashboard because you didnt have hundreds of thousands (millions?) of users refreshing to see new content, querying across dozens of databases to see likes, follows, etc. It was very common to frequently refresh tumblr to see the latest content.

Imagine had they cached this data instead and only updated it based on a lastUpdated fields value being different.

I think the "db in browser" idea is interesting because it makes the user pay for the compute for many more of the typical DB tasks.

Any time you're in a situation where you're in an aggregation point you need to be wary of Moore's law; this is load balancers, database systems, log aggregation, etc.

If your computers are getting N cheaper / faster per year, but you're getting M new clients and all your clients are also getting N cheaper / faster per year, you're going to be getting more traffic faster than you can scale, unless you figure out how to cheat. Cheating may be sharding your internal workloads, but it may also be "make someone else do the work."

That analytics in the browser is about 10000x times more performant, and doesn't contest on a shared resource.

How would DuckDB know which *.gz are in that folder? Is there a directory listing?

IndexedDB is a key/value store; DuckDB is a relational analytic database. You would use IndexedDB for something like application data, and DuckDB for (e.g.) a data-heavy dashboard or ad-hoc query tool.

Per [0], surprisingly, you can beat IndexedDB performance by layering another DB engine on top. Discussed on HN: https://news.ycombinator.com/item?id=28156831

[0] https://jlongster.com/future-sql-web

If you use https://dexie.org/ on top of IndexDB, you can turn it into a fairly useful document store with secondary indices, an improved query building interface, and reactive queries. It's doing exactly what you would expect on top of IndexDB if you did it yourself, but saves a ton of boilerplate. They have a sync service to go with it, but in my usage I managed that myself.

IndexedDB should have been SQLite, but for various squishy reasons it isn't, it's something less good.

Because of the IndexedDB API?

It feels very impractical indeed. Also the size of the binary to load the compiled wasm. All this would be much better done on the server and if really needed, users may be given a way to download results (ideally with their own preferred tool for fetching files)

Maybe you’re thinking of WebSQL? https://caniuse.com/sql-storage

Columnar analytics are slow in non-columnar engines

I have an example of a database hosted on GitHub: https://github.com/ClickHouse/web-tables-demo

Which you can query anywhere using a ClickHouse local database engine.

When you do a query, it downloads only the required ranges of required columns from the server and does computations on your machine.

In the same way you can query any data sources with ClickHouse, either local or remote, which is a strict superset in power and usability than duckdb.

The data would be sent to the browser

As far as I can tell they're not storing anything locally, they're pulling the data from Google Cloud Storage as you access it.

That said, a Wasm module doesn't have access to any storage facilities that the browser doesn't already expose (IndexedDB, OPFS, cookies, etc.) so even if it could be used by marketers, they would gain nothing by using DuckDB for that over just using the same underlying browser storage API.

I'd prefer that to page reloads, lost focus, scroll position and any input between clicking on a checkbox and receiving a response. Bonus points if I only have to download a db-diff next time, compared to downloading 5x db worth through POST /get-products?<filters>&page=<n> in every e-shopping session. Rarely a client-facing business database exceeds your average youtube cat video in size (assuming pics stored as urls).

Ten years ago your concern might be valid but in 2024, given that most people in the developed economies have ample amount of 4g/5g data plans with generous RAM/storage on their mobile phones, it makes more sense to just send them the whole DB once.

This is the same spirit as Electron vs native. Of course native uses less ram and is efficient but its not noticeable by the end user (because they dont care how its built). So we let developers write desktop apps in HTML/JS and end users do not notice usually until they open up task manager

We have abundance of data, memory, storage that continually drops in cost. It makes a lot of sense to simply download the DB once and then access it locally with optional syncing.

This is the future because we can't count on mobile phones to continuously maintain connectivity especially with limited data plans. Far better to simply allocate % of your fixed data plan and decide which local first app to install on your phone.

I think their tool is designed for working professionals on laptops - seems like analysts doing traditional business insights reporting. Perhaps there's a world where that target audience is using their phones, but I doubt it's a use case they're too concerned with.

I would be worried about this approach for something that doesn't need to expose a SQL interface though! It's not the right fit for most webapps.

If it’s an app you use regularly that trade off could easily be worth it.