Workflow design help (using APIs and AI)

I’m hoping to get some feedback on my approach to solving a client’s problem using Glide and specifically my current workflow.

Goal: I can press a button and get a list of leads I can call.

Yes that sounds easier said than done :slight_smile: Here is the detailed version:

Client sells non-gmo food ingredients.

  1. Specify an product category, e.g. energy drinks
  2. Provide a URL and search for similar companies using the keyword “energy drinks” (via exa.ai Find Similar API)
  3. Get specified number of websites that match query along with sub-pages from top-level domain based on “list company location, products and ingredients” as response. Result could be …com/our-products or …/store or it could be that each category of product has it’s own sub-page, e.g. …com/jelly-lollies, …/sauces…
  4. Click a button!
  5. Enrich the org data (using Apollo API)
  6. Use AI to check for the most likely place to start looking for products and their ingredients. Example: chobani.com.au/our-products, however depending on the site structure, this page may not be obvious or it might just be …com/
  7. Use Apify’s API to crawl the specified URL (that is most likely to contain product and ingredient info), call a Glide webhook and request data as JSON which is stored in a glide column.
  8. Use AI to read this response and identify a list of products and their ingredients.
  9. Take the non-gmo ingredient list my client sells and compare it to the crawl data/ingredients, and get a list of ingredient categories that a client uses, e.g. sweetners, starches, protein powder, as well as “[non-gmo ingredient] used in [client’s product, e.g. snack bar” so the sales rep knows what to discuss with lead. Using AI for this vs. logic, but I think I’d need to split data into arrays to get the lists I need.
  10. Find any people in specific roles at the lead company, e.g. R&D (using Apollo API)
  11. Find any news or other info about the company that could be a hook or opening discussion point, including any companies that the person worked at previously that my client has also worked with, based on organisation names in Pipedrive.
  12. Create a summary in Glide for each call using all the inputs above.
  13. Allow for call summary notes
  14. Generate follow-up email, edit and send using a Glide action.

:face_exhaling:

And that’s it! Steps 4…12 are executed via a workflow that is triggered at the push of a button.

I’m most interested in feedback on the AI-powered steps especially:
6: Not all website structures are the same so my prompt needs to account for this and fall-back to the high-level domain if nothing suitable is found, maybe a confidence score is needed in prompt?
8: Not as hard as my prompt returns JSON,
9: Hardest and most crucial: I need to know which ingredients to discuss and which product the lead will use the ingredient in, e.g. sweetners (used in Jelly lollies) and to get output in a form I can use to display lists.

I’m thinking these prompts might need to be document to text or something other than generate text but open to suggestions.

Here is a screen showing the steps broken down so I can trigger and test each step in the workflow, but they are actually all triggered by a Research button one screen up from this one.

I guess the final state will be that once the research step has been completed and the summary generated, then the status and/or button will update or be hidden and replaced with “View call summary” or similar.

Yes there are a lot of steps, but each one on its own is reasonably straight forward. As with all AI-powered workflows, getting the right data in and outputs out is key which is what I’m hoping you can share your collective wisdom on. Thanks!!!

Sounds like a nice flow. Most likely the part you will struggle most is the sub-pages. I understand it’s a must to try to pull all products and ingredients to match what your client is trying to sell?

Yes, it is to automate the manual research required to find products and ingredients usually on a website.

Any suggestions?

I think the only reliable option is to crawl an entire website and then use AI to find their products and associated ingredients.

I have run into the product page having different names like menu, recipes, solutions, catalogue etc because food and bev manufacturers all have slightly different offerings.