Any suggestions to have more rows beyond 25000?

I thought of developing an application for companies that use motorcycles to transport passengers. On average, a company carries out 150 transports per day.
A simple account:

150 x 30 days = 4500

So just a single company would take up 4,500 rows of data in the spreadsheet, and once it got to 25,000 rows, it would have no more space :frowning: .

Any suggestions to have more rows beyond 25000?

Use zapier or Integromat to add rows to a Google sheet tab that is not used by your app in any way. Then use ā€˜=filterā€™ in a tab that is used by your app to bring in data for the last ~30 days.

4 Likes

i would suggest google scripts and a web app, to deal with the big amount of rowsā€¦ and add webhooks to bring results to Glideā€¦ but that will slow down your App.

so the best solution is to support Glide App with Google web app in the Glide webview :wink: ā€¦ this option is blazing fast! and works on multiple users accessing the same data.

3 Likes

As @Eric_Penn alluded toā€¦how long do you need to retain those old records? If itā€™s indefinitely and you no longer need them in the app, then yes, you may need a background process to move the data to an unused sheet.

If old records can be deleted, then I would maybe consider something like this:

2 Likes

I thought about using Google Apps Script to move older data and then implementing some Python lookups to return information to the user in the browser. The idea is to use Glide to validate at first and then use actual programming for something bigger.

Hola,

Here is another idea to support your historical data using a GS as an API https://community.glideapps.com/t/a-google-sheet-working-as-a-rest-api-good-or-bad-idea/

Saludos

@Jeff_Hager As we know, the limit of 25000 rows forces us, above all, to adopt solutions such as ā€œmake row ownerā€ to reduce the amount of rows to be synchronized in the device and not to lose performance.
But do we have an absolute effective limit on the amount of rows that can exist in the table?
Assuming you have a 100,000 row users table (GT or GS), and an additional 300,000 row filtered table with ā€œmake row ownerā€ (where each user would download no more than a thousand lines), would that work?
In short, is there still some technical limit of Glide or even in terms of license?
I have never tried with these large numbersā€¦

If you are looking for hard number, here are the known limits for sheets and tables themselves.

https://community.glideapps.com/t/row-count-wrong/21515/33?u=jeff_hager

I donā€™t think we could ever reach the maximum limit of a sheet or table without completely bringing the app to itā€™s knees. Largest Iā€™ve ever tried was 120,000 rows of simple data (couple of columns and no computed columns). I also attempted to show all 120,000 rows on the same screen. That alone pretty much killed my phone and worked my laptop pretty hard.

As far as Iā€™m concerned, the main limitations are internet speed, cache storage, and RAM/CPU speed. How fast can you download thousands of rows of data? How much cache and RAM storage does your device have available to store and retrieve that data? How fast can your processor spin through all computed columns to run calculations? How much are you rendering on the screen at one time? How much html and javascript needs to be dynamically generated and rendered on the screen? How many images need to be downloaded, cached, and rendered?

Thereā€™s still the limit (bottleneck) of glide and google having to synchronize data between themselves before any data synchronizes between glide and the end user. Thatā€™s a whole extra path that the data needs to travel. Even with row owners to limit data to the end user, the entire database still needs to travel to and from the google sheet. In cases like this, using glide tables exclusively or even partially will help.

The effective limits are when when the app stops working effectively. That limit will be different for each and every app. My app doesnā€™t have what I consider a lot of rows and works great, but I have one tab that triggers an intense amount computations and relations, which causes the app to freeze for a few seconds while itā€™s loading. After that, itā€™s fine. Itā€™s not the number of rows thatā€™s limiting me. Itā€™s the amount of work that Iā€™m requiring my device to perform against those rows.

So many factors come into play. Results will differ per app and per user. I canā€™t theorize when someone elseā€™s app will hit a limit. Thatā€™s just impossible to do. My recommendation is to just create a massive amount of test data. Test, test, test. Push the app to the limits and find out what does or doesnā€™t work.

Just realize that the end userā€™s device is downloading and caching a duplicate of the entire database (minus unowned rows), and doing all of the computations and calculations, and obviously rendering the screen. Thatā€™s your limitation right thereā€¦the capabilities of the end userā€™s device.

Iā€™m sure Glide takes a bigger financial hit (as far as storage and bandwidth costs) when the dataset is large, but thatā€™s averaged out among all users. If it ever got out of control, Iā€™m sure glide would be having a chat with that user. Iā€™m guessing that glide does not strictly enforce the 25k limit because, with the way the glide infrastructure works, you are going to run into several other bottlenecks before even coming close to a hard max row limit. If/when they change how the data is handled on the back end, then I would be willing to bet that they will have tiered pricing based on the amount of data neededā€¦and the app would no longer be caching entire datasets, but instead it would lazy load data as needed. But, that would require them to completely redesign how glide works now.

4 Likes

I canā€™t access :point_up_2:
ā€¦ anyway thanks Jeff, for your exhaustive explanations and all the time you have dedicated to the answer.
Yes, certainly many factors that can affect performance. My curiosity, before experimenting with myself, was to understand if there was a predetermined technical limit or a license-level restriction.
Thanks again

1 Like

Hmm, this is what was written in the post I was referring to. Maybe itā€™s not accessible because itā€™s a closed topic, but I can see it as a moderator??? Not sure. Anyway, hopefully the explanation from David below helps answer your question.

Because we charge based on the value of our product and the cost of our service, not based on the theoretical limits of our servers.

Glide Tables can scale indefinitely, both in terms of storage capacity and cost ā€” they do not scale indefinitely for free.

A Google Sheet has a hard limit of 5 million cells. A Glide Table can scale to 5 trillion (for example), at some high cost.

The 25k row limit is not currently driven by Google Sheetā€™s storage limitations anyway. Itā€™s a memory limit and transfer limit in the app client.

In the future, we will offer higher row limits. Apps that need millions of rows will not be able to use Google Sheets, because Google Sheets cannot scale that large.

2 Likes

This is the message:

Oops! That page doesnā€™t exist or is private.

Closed topics are technically supposed to be deleted after a few weeks of being closed. Maybe they are not deleted, but instead marked as private. Thereā€™s also private categories that not everybody can see.

The biggest clue is that when I post a link to another post, if a preview doesnā€™t generate to replace the link, then thatā€™s a clue that that post is private or doesnā€™t exist. I need to be more aware of when that happens. Itā€™s easy for us moderators and experts to miss, since itā€™s not obvious when a post isnā€™t visible to everyone.

:see_no_evil:

1 Like