Scraping website data directly to Glide table

Ok, bear with me here as I am a beginner at these types of things. Currently I have a few columns in my Google Sheets table that are populated using importxml formulas. However, the columns are dynamic and change with choices from the user. As you can imagine, the load time of those columns into the Glide app is way too long. So…

Is there any good way to pull in web data to Glide columns similar to the importxml method? I know many new experimental columns have been added, like Fetch JSON. Could something like that do the trick? I know next to nothing about JSON, but I’m a quick learner if someone cares to teach. :slight_smile: The site I’m trying to pull data from is https://barttorvik.com/team.php?team=Indiana

I’m wanting to pull in data from the schedule table near the bottom of the page. Anyone have any great ideas on this? Possible or No?

Adam's WI Sports Blog: Sacred Data looks promising. You could probably get the data you need as JSON and use the Fetch JSON plugin to pull it into Glide. Massaging it into a format suitable for displaying in your App might take a bit of work, but would definitely be doable.

Yes, I’ve looked through that page and tried putting a couple JSON URLs into the Fetch JSON plugin, but keep getting errors. I have no idea how to use the JQ Query field to access the data I’m looking for. Like I said, I’m a complete newb with JSON, so I’m probably doing something wrong. For example, if I look at this JSON, https://barttorvik.com/2023_super_sked.json ; what is the proper way to use JQ Query to access certain parts of that file? It just looks like a big comma delimited list to me, with no other tags or classes. Is that how a JSON file is supposed to look?

this json is too big for single Glide cell. Im allredy pasted it and can navigate, but RAM cost rase from 4GB to 10GB. Another case, when i had copyed apps, then the cell content didnt copied, it means this json is too big for glide. You need to find the way how to reduce query range of json file
May be
about jq for this json: .[index_of_element][index_of_element_field]
you can use jqplay.org to test it

Yes, that’s valid JSON. But as @slscustom.ru pointed out, it’s too big for Glide to handle. That JSON file is 1.1MB, which exceeds the Glide limit for a single cell (1MB).

I’d suggest reaching out to the site owner - he seems quite approachable. Most likely he can give you a smaller filtered subset of that data.

1 Like

ok, thank you. I didn’t realize there was a cell size limit like that. Yeah, I’m sure he probably already has a json that’s more filtered, or could create one on request. The one I posted here had way more info than I needed, but I figured I could still use it for testing. Now that I know it’s too big, that makes sense. I appreciate the help! If I can get this figured out, it opens up so many other possibilities to improve and speed up my app!

Once you have one that Glide can handle, feel free to ask if you need help parsing it.

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.