More than 25k rows in the sheet

[kunksed]

I am developing an app around COVID19 which may end up having 25k+ data points. How do I get Glide to extend the limit on the Pro version?

[Mark]

The 25k limit is a technical limitation, not a number we can easily increase. Could you tell us more about your app, and about how much data it will have to hold?

[TimNZ]

I enabled Comments on certain sheets,
and I guess 25k could be hit quickly if the app got popular?

[Jeff_Hager]

The app:comments sheet is only a log. I assume Glide does not attempt pull any of that data back into the app.

[Aristide_Salvatore]

And what happend when the app reach this number of rows? Its a limit from google sheet or from glide?

[Mark]

There are bottlenecks both in Google Sheets as well as in Glide. The app will get slower and it will take longer to reload from the spreadsheet, until at some number of rows it won’t reload anymore.

[Aristide_Salvatore]

Thanks very good

[mthakershi]

So @Mark, say I have one anchor sheet and 5 dependent sheets pointing back to the anchor sheet like FK to PK in RDBMS. If I have about 5K rows in the anchor sheet and 20 rows for each of the anchor sheet row - totaling 5K * 5 sheets * 20 records for each anchor row = 25K * 20 = 500K

Does this bottleneck apply to this scenario? My App filters out few anchor rows that are applicable to the signed in user and then user navigates to the dependent sections from there.

Thanks for your guidance.

[Jeff_Hager]

It may or may not work. I think you will see a slowdown, even if each relation only shows a handle of rows, because I think the phone will be caching some of the data as well and has to download it. I think the biggest slowdown is Glide having to sync up all of that data, which can take some time. I ran a simple test with up to 120,000 rows in a single relation, so that’s a bit extreme of a case. Smaller relations worked better though, so breaking up the data will help, but it did work my computer and phone pretty hard.

Sheet with 24K rows to start with

I ran a stress test on one of my apps. I had 5 columns and 120,000 rows to start out with. This was loaded into a single list. While it did load everything…I think…it really pushed my computer and phone to their limits. Refreshing the sheet in the builder took a few minutes to load the entire sheet and the builder was sluggish, although that could have been the entire browser being slow. I’m sure the browser is caching all of this between the sheet, the builder and running a published versi…

[Mark]

500k rows will almost certainly not work.

[jpaul156]

I tested mine with about 100K rows, and it worked. Though a bit slower.

[mthakershi]

E.g. I am expecting 100K users for my App. And there will be plenty of dependent rows for each user. I am just exploring ideas.

1. One app is going to slow things down and if I am making enough profit, I am not going to cut my infra short. So can you sell an App pool? I buy 10 PRO apps but it is transparent to the user. In this case, data store is still one google spreadsheet. I don’t know but you probably do something like this in your top tier behind the scenes.

2. If I have a way to segment my user population beforehand and tie them to a specific instance of my App then can that create issues? E.g. I create 26 PRO apps with 26 sheets having exactly same structure and UI. If user’s email starts with letter ‘m’, somehow I make sure that user always gets endpoint ‘m.myapp.com’ and so on. However, in this case, if there is a need to share data between users in different data stores.

Any other suggestions or critique is welcome.