Is it possible?

I have a tool that scrapes data into an excel database. the data is sent to a live feed via a webhook and the data is also aggregated into a dashboard, where we analyze the data and visualize certain metrics.

Currently, we have one dashboard and we manually run it as a total aggregate on the complete data set. But we would like to build a mobile/web app that would enable us to allow for user customization at the data set level without affecting other users, or the universal data.

Essentially, we would automate the dashboard, but we would like to enable users to create a profile and then select from the global data set what specific data is presented on their dashboard/live feed.

For example, if we were showing temperature (weather) data for all cities in the USA, the primary feed and dashboard would have every single city’s data point popping up in the live feed as new data hits the feed, as well as a dashboard that represents the total data set. If a user wants to isolate data for Chicago, New York and Miami, we would like to allow them to select this data and have a custom dashboard and feed that represents their choices, without it affecting other users’ selections/the global dashboard.

this tool is client/customer-facing, so, it’s important that user/profiles are not limited.

currently trying to find solutions for this, and wondering if this is possible with glide? if so, how?

TIA!

Absolutely doable with Glide.

The only thing that I can think of that might be a challenge for you is the number of data rows. Glide currently has a soft limit of 25k rows. You can exceed this (it’s currently not enforced), but as you do the performance of your app will suffer. This is because Glide doesn’t use a traditional client-server architecture - instead, all data is downloaded to the end users device up front, and then only changes are synced to the back end. This makes Glide apps very snappy, but comes with an obvious downside. I’m aware that Glide are working to extend this limit, but there is no timeline that I’m aware of.

All that said, there are strategies that can be adopted to work around the limit - moving data in and out, archiving older data, etc.

But anyway, I’d say definitely give Glide a try. One thing you’ll find is that once you get your head around the basics, you can very quickly prototype ideas and come up with some very slick looking user interfaces.

And, Glide is a bunch of fun to work with :wink:

Awesome, Yeah, I imagine that we would be okay archiving after 25k rows without effecting the use. is there a way to automate this inside glide?

Also, would it be 25k limit for the main data base, or would users data be included? for example, is the global data base has 20k data points, and the user selects 10k of them, would that ultimately result in 30k data rows? or how would that work?

Do you have any recommendations on where to start with, ie. understanding the multi user interface and customization?

Glide has an API that you can use to add/remove/change data, but if you’ll be using Excel as a backend then I’d imagine you’d be able to automate it there? (I’m not an Excel user, so don’t ask me for specifics :stuck_out_tongue: )

Any row that is referenced anywhere in your app is counted. And no, it’s not per user. 20k rows is 20k rows, whether you have one user or one million users.

A few pointers off the top of my head:

  • You’ll want to familiarise yourself with User Specific Columns. These are very important when it comes to personalising the user experience. Creating things like custom dynamic filters, charts, tables, etc all require the use of USC’s.
  • On the topic of charts, if you expect to be doing a lot of charting I’d highly recommend quickchart.io. Although Glide has a native chart component, it’s quite basic and you’ll very quickly find yourself bumping into its limits. Also check out this thread for some inspiration.
  • Also, try and resist the urge to do lots of calculations in your backend spreadsheets. Glide has a very rich set of computed column types and plugins, including an Excel plugin column that supports all your favourite Excel formulas. Doing all your calculations in Glide makes your apps perform much better, as everything is done client side and you eliminate any delays waiting for data to sync between the client and the back end.
  • This community is awesome, and a great resource. Check out the #tutorials section, and pay particular attention to anything from @Robert_Petitto or @darren (not me, a different Darren). Both of those guys also have their own Youtube channels, with a bunch of Glide related tips and tutorial videos.
4 Likes

This is great, thank you!

question on this bit though, what is the delay time for sync between client and backend. our data is near real time, and its important we keep it that way as best as possible.

oh, and here is my personal list of things I wish I knew when I first started out with Glide… :wink:

1 Like

awesome, thanks!

I’m not sure about Excel, but when you’re using Google Sheets as a backend the delay can be anywhere between a few seconds and a few minutes. This only really becomes a problem if you’re relying on the results of formulas in your backend sheets. For example, a user enters their date of birth, and you want to calculate their age and show it to them. It’s very easy to do this with an Excel or Google Sheets formula, but doing so would result in a pretty crappy user experience, because the data needs to travel from the client to the Glide back end servers, from there to your Excel sheet, wait for the calculation, and then go all the way back again. A much better way to do this is to use a simple Math computed column in Glide. Then the computation happens on the user device, and the result is available instantly.

All that said, for best performance you may want to take a look at Glide Tables.

okay, got it. well, in our specific situation, the user would not be giving any data to the system. all of our data is provided in the back end from our tool, and then analyzed. in this instance, would it still be beneficial to do the calculations in glide, since the data would not need to travel back and forth from the users to the back end?

I.e. the data will travel from our scraper → database-> calculations → user interface/dashboard.
Data doesnt need to go backwards. unless this would be a function of the user selections?

global data scrape->gd database-> gd calculations-> user IF/DB with user level customization being a data filter essentially.

If you’re already aggregating/analysing/summarising data in the back end, then you probably won’t want to change that - and I don’t think it would make sense to do so.

I’d say just keep my comments in the back of your mind. Any time you need to collect input from a user and “do something” with it, always look first for a way to do that in Glide.

1 Like