User-specifics columns are not a security feature (well, you can is some way, but it is not the purpose.
Row owners is the way to protect data from unauthorized access.
Visibility and filter conditions are only applying to the front end. Meaning that even if you add a visibility condition to allow only admins to see data, if you are not using row owners, the data is still downloadable.
Take a look at this thread: Security tip: Managing role-based permissions in Glide apps