I know this is a discussion point that circles endlessly so apologies if the answer is buried somewhere already.
Quick question though.
Let’s assume the following parameters for my app:
5000 users
20000 rows
Its an event/calendar based app so the users only need to see and interact with approx 500 rows at any given time (because the majority of the data is in the past and therefore no longer relevant other than reporting and stats)
Loading to the screen only ever includes a max of 50 records from an event table
So I guess the question is……even though I’m approaching the max number of rows, will the performance remain quick as I’m only referencing and loading a percentage of the rows to the screen? Or is the concern around speed related the number or rows being searched by the google sheet api?
Hopefully that makes sense? I’m just wondering where the bottleneck is in the scenario above? Or is there one?
None of the above really matters in terms of how much data is loaded on the end users device. Think of it this way - if a single row in a table is referenced by your App, then all rows in that table will be loaded.
So even though your users might only be interacting with 500 rows, they’ll still have all 20,000 loaded on their device.
Two ways you can deal with this.
The first way is to apply Row Owners. With row owners applied, user devices will only load owned rows. Depending on the nature of your App, this can have a dramatic impact on the total number of rows loaded by each device, with a resultant improvement in initial loading time.
The second way is to move older data out of your table. Either delete it, or move it to some other table that isn’t referenced by your App. You mentioned that you want to keep older data for reporting and stats. Perhaps you might want to consider a second App just for this purpose, and keep your primary App purely transactional. Or maybe you can consolidate older data - roll it up by month or similar.
As already mentioned, no. If you leave it as it is, you can expect to see things start to slow down as the total number of rows approaches and exceeds 25k.
This is definitely a factor. One of the reasons for the 25k limit on Google Sheet connected tables is that there are limits imposed by the Google API that makes it difficult for Glide to reliably sync sheets with row counts above that. Row count is not the only factor. The number of columns also comes into play, as does the amount of data stored in each column. So it’s more a factor of the total amount of data, as opposed to a fixed row count. For example, I’ve been able to successfully sync Google Sheets with in excess of 100k rows with Glide, but those sheets only had a handful of columns. YMMV.
Yes, Glide have announced that they will introduce “large” Glide Tables sometime this year, that will support up to 10 million rows (and probably more).
We don’t have full details yet, but I expect they will most likely come with some caveats. It’s not practical to load 10 million rows on a user device, so that means that these large tables will almost certainly use server side computations. There might also be some types of operations that might not be possible on these tables - rollups, for example. This is just speculation - I know none of this for a fact. We’ll find out soon enough.
Last question I promise……there’s currently no quick/easy method of converting google sheets data to glide tables? I remember looking at it previously and there was no native method. I saw a new csv upload option so maybe that would work?
The reason I’m digging on this now is multi faceted:
The app I have has organically grown over time as I’ve learned more and more. It was never really intended to be where it is so I guess I need to decide what the end goal is
With the new price structure it moves from a hobby to something I need to consider a little more seriously as €100 a month isn’t cheap as we move forward
There’s an appetite for the app but I’m throttling the number of people using it because i need to know it’s got a long term life (5 - 10 years). It would be disingenuous to sign folks up and then pull the rug from them in a year when there’s a dependency built up by them.
Ultimately, I see the way the glide product has grown and improved in 2 years and I’m hopeful that any barriers to the success of my app will be overcome with time as glide layer out capability.
There are plenty of ways to move the data, but you’re still left with the issue of adding all the computed columns and reconfiguring any attached screens and related actions, etc.
If you’re in no hurry, I’d advise waiting a bit. I suspect that some time in the not too distant future Glide may provide some sort of migration tool to make this job a bit easier. I’ve been told that it’s technically feasible, it’s just a matter of it bubbling to the top of the priority list.
@Darren_Murphy if you’re accessing rows through a relation on another table then does your device still load in all rows or just the relevant ones in the relation?
The only thing that changes the above is the application of Row Owners.
Just to expand on this a little. This behaviour makes sense once you understand that relations are computed columns, and all computed columns are “computed” on the users device. So when a user opens an App, all the data that they have access to is downloaded, and then all computed columns are built.