Working away at getting a demo going. So far, I have the IR sensor mapped to an additive synth. I have the amplitude of the partials randomly changing to give it more texture. It was tricky figuring out how to make it so changing the amplitude of each of the partials didn't make pops.
I'm working at getting the colour sensor to control the speed of which the partial's amplitudes change. I was trying for the longest time to get the colour sensor mapped onto a [swatch] object, but the saturation kept going down and I couldn't figure out how to make it steady. I instead mapped it to a [jit.matrix] and doubled the brightness with a [jit.brcosa]. When I place any colour in front of the sensor, it shows in the pwindow. But the tricky part is to have useable data to represent each colour in order to use specific colours to trigger other things. I ended up doing a comparison of all three colours in order to tell if the colour is either red, green, blue, or yellow. It's a bit clunky, but for my purposes (for now) it works. Purple appears too similar to blue and orange appears too similar to yellow to be recognized in this manner.
Alternatively, I could have used some sort of machine learning to do this, and perhaps be able to differentiate between the whole rainbow (or perhaps not). I should keep this in mind for the future.