I was doing some testing with both paint chips and with fabric that had fabric paint on them. I worked on getting the patch the most accurate when using either method and while using all colours available. It was really tricky for it to recognize the difference between orange and red, but I think I have it as close as it is going to get now. I noticed that the "clear" threshold is best set to 850 while using paint chips, and 1000 while using the fabric paint swatches. It felt like the gloves with the paint swatches were more accurate than the paint chips, and being able to use a higher threshold probably contributes to that.
So, now that I've done all that testing, I will add orange and purple paint to the backs of the gloves that I made. I feel like have three colours per hand on the palms would be a bit too much. Also, I don't plan to use orange or purple that often, so that makes it more convenient for when I am not using those colours at all.
Version 5: https://github.com/CLKo1/colourdetect-maxmsp
If you are looking for a summary for my Masters thesis, it is here.