Archiving to Big Tables

I’m considering building an archiving feature where rows (each row is a project) are sent to a big table equivalent of the regular Glide Table for archiving purposes.

The reason I’m doing this instead of a Boolean & filter on the regular Glide Table is that over time there will be more than 25,000 rows created, and simply deleting them isn’t ideal as we want to preserve the ability to reopen a project down the track.

Wondering how people generally tackle this, pros and cons they have found, and also how you might handle searching for projects in this case…Is there a way to search both active and archived projects at once even though they’re on different tables?

1 Like

If you don’t have that many fields to search on, I would recommend a custom search, where you let the user type into a user-specific field, and then do more work on the backend to check if their “search” term would return records. Show those records in 2 collections, one for the active table, one for the archived table.

1 Like

What’s preventing you from only using a Big Table?

1 Like

It may be possible. I’m just cautious knowing there are some limitations with Big Tables. The Glide Table of projects feeds to jobs, tasks and then bookings.

I don’t think I can build all of it in Big Tables - there’s another Glide Table containing a date in each row (0 to 90 days from current time) with some functionality that involves calculations from multiple relations to automatically determine the next available dates and staff members that can be booked for either field or office work, based on their schedule, leave days, current bookings, technical capabilities, etc.

I could probably do some sort of hybrid approach where bookings, leave, etc in the future that must be used in these calculations are in regular Glide Tables, and everything in the past is stored in Big Tables for archiving, analysis, etc. This could get complicated.

It is partially a Big Tables learning and discovery issue where I’d need to spend time working it all out, combined with the fact that the app is already built.