- Shane Guffogg is a multi-media summary artist with synesthesia, which means he “hears coloration.”
- Guffogg labored with AI consultants and musicians to compose music that corresponded together with his work.
- He believes AI remains to be a instrument that “wants oversight” however its enhanced his inventive course of.
That is an as-told-to dialog with Shane Guffogg, an American artist who launched “On the Nonetheless Level of the Turning World – Strangers of Time,” an exhibition of 21 work on the Venice Biennale earlier this month. This dialog has been condensed and edited for readability.
I’ve synesthesia, which suggests I hear coloration.
So, what I am listening to once I paint is necessary. I take heed to Indian classical music, Gregorian chants, and a few obscure composers reminiscent of Gyorgy Ligeti, Leo Ornstein, and Terry Riley. The music sparks my creativity and permits me to be utterly current and in that second.
For years, I have been preoccupied with what my work may sound like. The AI revolution pushed me to seek for consultants who might assist me. My first level of contact was Radhika Dirks, an AI and quantum computing skilled. We had a few Zoom classes, and he or she informed me — to the very best of her data — that no AI program might assist me. As an alternative, she prompt I create a visible alphabet that matched the musical chords I heard in my thoughts to colours.
I believed it could possibly be a option to propel my creativity. It additionally constructed upon the concept of an unconscious alphabet that has knowledgeable my artwork all through my profession.
I met with musicians and AI consultants to create a visible alphabet
I began by searching for musicians to collaborate with and met Anthony Cardella, a younger, extremely gifted pianist in Los Angeles. He is a PhD pupil at USC and occurred to know — and even play — many obscure composers I take heed to once I paint.
We began collaborating. We might sit down and look at my work collectively. I might zoom in on a coloration in Photoshop, take a look at it, and sensorially really feel the musical notice. Then I might inform Anthony. I might say, for instance, I believe that is the colour of the notice B. He’d hit the B, and I might say, “No, that is not it; strive a B sharp?” After just a few trials, he’d immediately hit the appropriate notes. I might know as a result of the colours would start to vibrate for me. Collectively, we have charted chords that correspond to 40 colours.
Quickly after, I met an AI researcher named Jonah Lynch via mutual contacts. He works on the intersection of the digital humanities and machine studying. I invited him over to my ranch in central California and defined the work I had been doing and the way I created my work. We had lengthy discussions about artwork, poetry, and creating an AI algorithm that could possibly be fed the chords.
He developed a program to “learn” my work and convert them into music. I gave him the principle colours I utilized in every portray and the chords I hear once I see these colours. Jonah watched movies of me portray, studied the motion of my fingers, and wrote software program that sampled photos of the work, following my hand actions, and assigned every coloration sampled from the work to its corresponding chord. Then, he fed this sequence of chords right into a neural community that has memorized a lot of the final 500 years of keyboard music. He prompted the community to “dream” of recent sequences primarily based on the color-chord sequences and the historical past of Western music to create pages of sheet music.
Once I heard that music performed again to me, it introduced tears to my eyes. It was only a tough model of what I heard whereas portray, however I believed, “There it’s.”
I took the music again to Anthony, the pianist. Amazingly, I might level to the sheet music and inform him what compositions I used to be listening to whereas portray, and he’d say, “Sure, I can see it within the chords.” The Indian ragas, the Gregorian chants, the Ligeti, and Ornstein — they had been all there.
Nonetheless, the music was largely a sequence of chords at that stage. Anthony stated we might have melodies if we rearranged it a bit.
AI remains to be a instrument that wants human oversight
Shane Guffogg
We composed music for a number of work and have performed it for audiences worldwide. We held a live performance final month on the Forest Garden Museum in Los Angeles, the place I additionally had just a few work in a present. The viewers might take a look at the work whereas Anthony performed, which was a profound expertise. A few folks cried.
On the launch of my newest exhibition throughout the opening week of the Venice Biennale, Anthony performed the world premiere of a sonata he composed impressed by my portray, Solely Via Time Time is Conquered, to a dwell viewers. After the efficiency, I talked to a number of folks, and so they stated they might see the place the colours and the notes met on the portray. It was one thing that they had by no means skilled.
I do know many individuals are very afraid of AI, and I, too, see it as a instrument that wants human oversight. It isn’t a way to an finish. Nonetheless, it opened up many prospects and enhanced my inventive course of. I do not know if I might have unlocked the musicality in my work in an actual manner with out it.
Hear the sonata beneath:
