Need Help in removing duplicates after entries are coming from Make

[Dilip_Adiga]

Hi guys,

I am using Make.com to fetch some data on a daily basis. I fetch data which basically inputs Text in a column and Other data.

Since its done on daily basis sometimes same text messages are copied into the database. I want it to remove duplicates when added. Is there a way to handle this in glide automatically? Without having any user intervention as soon as make write data to glide table??

Thank you in advance for your help.

Regards,
Dilip

[Robert_Petitto]

Is there an identifier for each item? If so, I’d add a query rows module in your Make Scenario to see if the duplicate has already been added. If so, filter that one out before adding the new ones.

[Dilip_Adiga]

Hi @Robert_Petitto ,

Thank you for your reply. Unfortunately currently the API response is not giving me any identifier.

I used combination of this solution from @Jeff_Hager

Is there a way to filter duplicates from an inline list?

This is what I would do if you wanted to keep everything within glide. First I would recommend having a RowID which will give every row a unique value. This will become important in a little bit. Then I would identify a unique value for each contact, whether it’s a name, phone number, email address, or something else that you know will be in each duplicate row, but is still unique only for that specific person. Let’s assume Name for now. Create a self relation that links the contact’s Name …

and by configuring workflow

PS

Since I had Beta access to scheduled workflow I could test it

I think the bigger issue in this would be when the table size increases and duplicates increate, each delete costs me an update, which becomes an issue.

Regards,
Dilip

[ThinhDinh]

@Dilip_Adiga

Thank you for your reply. Unfortunately currently the API response is not giving me any identifier.

Then how do you know it’s a duplicate when you add it to Glide later?

[Dilip_Adiga]

Hi @ThinhDinh

Thank you for your reply. Unfortunately I have to now rely on crude method of comparing texts

I.e. the text column is compared with self to check whether it is duplicated

Since there is no identifier coming in the api I guess this is the only option I have?

Regards,
Dilip

[ThinhDinh]

I mean what “text column” are you referring to here? Is it part of the API response?

If not, how do you know a row is duplicated or not later on?

[Dilip_Adiga]

Hi @ThinhDinh,

Yeah the text is part of api response. Everytime the api is queried I get text in response which I write to my text column of the table where I store this response.

The api just sends me entire response everyday meaning it will send older data plus new data everyday when queried

So hence in my table old response plus new response gets added meaning the “text”

Now my problem is the text column which is storing the response it gets from api since it’s written everyday there will be duplicates right I need to remove them or let api not write the existing values in the table again from api

PS I use make to call api and write response into glide table

Regards,
Dilip