using Photoshop to process sound

Moderator: interruptor

Post Reply
User avatar
interruptor
Posts: 1151
Joined: Sun Sep 04, 2005 11:46 pm
Location: Switzerland
Contact:

using Photoshop to process sound

Post by interruptor » Tue Jun 25, 2002 4:47 pm

I am not shure yet if this guy is trying to fool the world - but his idea is definitely tempting! Let's use the image processing tool photoshop to manipulate wav-files: <BR> <BR>--=--=------=--=------=--=---- <BR> Painting Sound <BR> By: Eric Hamilton <BR>----=--=------=--=------=--=-- <BR> <BR>Some people go to great lengths in the search for new sounds. As a teenager I was inspired by tales of Jimi Hendrix constantly looking for new ways to mangle his guitar sounds, and the accounts of electronic bands like New Order piping everything you can concieve through their Moog's filters. One day I was reading an issue of Mix Magazine when I saw an ad for a new type of synthesis program. It was a program that actually allows you to paint sound. I got to thinking, and I figured out that Photoshop has a raw file import feature, and Soundforge has a raw file output feature. I put two and two together and started to experiment. Here's what I figured out about painting sound. <BR>The hardest part is in the conversion. There's no way to achieve predictable results unless you can export an audio file, import it into photoshop, export it again, and import it into soundforge without changing the sound. There are actually several ways to accomplish this feat, but I will make it as easy as possible. <BR> <BR>Start by loading a sound into soundforge. Select File->Save as in the drop-down menu. You'll be presented with the save as dialog box. Change "Save as type" to "Raw File &#40;&#42;.raw, &#42;.&#42;&#41;". Give your file an apropriate name and save it in a directory where you'll be able to find it again easily. I created a sub-directory in my sample creations directory called, "photoshop", but as long as you can find it, anything will do. Next, you'll be presented with a dialog asking how to save the file. Select 8 bit, unsigned, mono, and click "Okay". <BR> <BR>The next step is to load the file into photoshop. Choose File->Open from Photoshop's drop-down menu, and you'll be presented with the Open dialog box. Browse to the directory where you saved the raw audio file, and make sure the "Files of type" select box is set to "All Formats" or "Raw &#40;&#42;.RAW&#41;". Select the file, and click "Open". Photoshop will pull up the "Raw Options" dialog to collect some information about your file's dimensions, channels, and header info. Normally, it will make a guess at the dimensions automatically. The dimensions that photoshop guesses at have always worked well for me, but once in a while the math doesn't pan out. If this is the case, photoshop will present you with blanks. Rather than trying to work out the math &#40;which will force you to cut out part of the sound&#41;, you'll want to insert silence in soundforge to create a good starting point for photoshop. You may have to experiment a bit to get this right, but here's a tip: It's easier for me to work backwards. When photoshop gets stumped, create a new file &#40;greyscale&#41;, and take a guess at how big it will need to be to handle your sound. 500x500 in greyscale is aproximately 5 1/2 seconds of audio. Save the file, import it into soundforge, zoom out so you can see the entire file, press the End key, and check the duration. If it's not long enough, try creating a larger file. If it's too long, that's fine. It'll work. Insert silence into your sound file until both files are the same size, re-save the raw audio file, and import into photoshop. Make sure it's set to 8 bits and no header. Click "Okay", and you should see an interesting greyscale image with some cool patterns. The top of the image will likely contain some blacks and whites, with a gradual fade as the image gets more and more grey towards the bottom. Saving the file from photoshop isn't such a hastle. It already knows what it needs to encode the raw file. I use <BR> the save-as dialog and choose a new name for the modified file so that you don't have to do the conversion all over again if you don't like the <BR> results. Once it's saved, load the file back into sound forge and compare with the original. Use the same settings for your soundforge file open dialog &#40;it'll bring up the "Raw Options" again&#41;, and you should be set. <BR> <BR>If both files sound the same, you're on the right track. Now let's take a look at what you can do to a sound in photoshop now that you know how to load and save. The sound maps to the image in the following ways &#40;from what I've been able to discern&#41;: Each greyscale pixel represents one sample. If the sample is dark, it will be low in amplitude. If it is bright, the amplitude will be high. Thus, the contrast of the image represents the over-all amplitude of the sound. Another interesting note is that Photoshop does not interpret negative amplitude as a negative color value. Instead, everything is positive, making grey the zero point in terms of audio. 128, 128, 128, to be exact in the RGB Value. <BR> <BR>The sound is mapped to the image in order from beginning to the end left-to-right and top-to-bottom. The beginning of the sound generally has the attack characteristics, and thus the most contrast in photoshop &#40;for percusive or plucked sounds&#41;. Here you may find you can do things with your sound's attack and decay properties that you never thought possible. For example, you can create a new channel and apply gradients running from top to bottom that will change the dynamics over time. You can twist and bend sounds in photoshop in ways unlike any other sound editor on the planet. To apply a gradient that will alter the sound's attack, open the channels palette, create a new channel, and name it "Selection mask". Then Set your foreground color to white, and your background color to black, apply a gradient from top to bottom, and <BR> re-select the original channel. Click Select->Load Selection, and choose "Selection Mask" from the drop-down menu. Click "Okay". Make sure your background color is set to 128, 128, 128, and hit delete. And you thought ADSR envelopes were powerful. <BR> <BR>The dimensions of the image have a large impact on the way that the audio will sound. Genarally a combination of height and width determines the length of the sound &#40;naturally, since more sample data means a longer duration&#41;. Something you might not guess right away though is that altering the width of the image &#42;can&#42; modify the fundamental frequency. The precise mapping for this is actually variable, though, depending upon the pattern of light and dark sounds in the image. I suggest you experiment, and use soundforge or another audio tool to tune the resulting sounds. Simply applying a filter can change the fundamental percieved frequency of your sound drastically. <BR>My main point here is that just as in real life, size matters when you're painting sounds. Altering your images dimensions will also alter the formants, the same way that playing a picolo produces different frequencies than playing a bassoon. <BR> <BR>There is a lot to be said for experimentation, but I must warn the beginner, photoshop is a powerful program that is not designed specifically to make audio files sound good. It does have a lot of <BR> features that have some very cool and interesting effects on your sound files, but be warned, it is very easy to create noise in photoshop that will not sound pleasant at all. <BR> <BR>For example, in many photographs and images, there is a high degree of contrast, and many pixels placed close to each other that are not similar in value. I'll call this texture from now on. In audio, there is generally much less texture. Texture's number one enemy in photoshop &#40;and your number one friend in fighting it&#41; is the gaussian blur located in the Filter->blur menu. Another way to fight texture is to reduce the contrast of the image, but this method is less effective in that it reduces texture by reducing the amplitude of your sound, which is not necesarily the result you're after. The gaussian blur does have some <BR> side-effects you may not be prepared for, though. In addition to smoothing out the rough edges and removing distortion, it will also alter the formant frequencies of your sound. This can be good or bad, and can be fixed or enhanced in soundforge to some extent with the graphic EQ feature - if you're good at identifying which frequency to boost or cut &#40;something I'd start practicing right now if you're not&#41;. <BR> <BR>For the most part, sounds you import will have blacks and whites up top, with a smooth fade to grey as it goes down. Images you might load will probably look quite different. This basic sound shape is a result of how sound works in nature: attack, sustain, and decay. That's why most synthesizers come with envelopes that let you edit Attack, Decay, Sustain, and Release values &#40;ADSR Envelopes&#41;. It's a good rule of thumb that unless you're planning to create a looping wave and apply ADSR envelopes in you sampler or soundforge, you should try to mimic the behavior in photoshop. <BR> <BR>Without altering the general shape of your sound, there are many things you can do that will wildly change or enhance the frequency spectrum. One of the most dramatic examples is the color curve feature in photoshop &#40;Image->Adjust->Curves&#41;. Also try Image->Adjust->Auto Levels to try to maintain a good center and maximize your signal. Each time you run the auto-levels command, you may need to re-adjust the center manually using the brightness/contrast dialog. To do this, I like to zoom in on the lower right corner of the image, use the dropper tool to select the last pixel, and then double click the forground color on the toolbar to view the color settings. If the RGB values are anything but 128, go into the brightness/contrast dialog and add or subtract to compensate. You may also want to adjust the contrast to slightly reduce the amplitude and avoid clipping effects. <BR> <BR>Some effects that have produced cool sounds are the Filter->Distort->Twirl effect, Filter->Render->Clouds or Filter->Render->Difference Clouds. You can achieve a strange blocky sound by running any plugin that has the effect of reducing the number of distinct colors in the image &#40;such as Cutout, Stained Glass, or Crystallize&#41;. <BR> <BR>Another thing you might want to try is the photoshop twist on sound morphing. To begin, choose two sounds that are roughly the same frequency. Load them both into Soundforge and add silence into the shorter sound until the two sounds share identical length. From there, import them into Photoshop with identical parameters. There are multiple ways you can morph the sounds. My favorite is to copy the shorter sound into a new layer on the larger sound.. position the smaller image so that it comes in where you want it to. Adjust the new layer's opacity to let some of the other sound mix into it. Now set up a selection on the smaller sound, and set the selection feather value &#40;in the Selection menu&#41; to something like 30 or 50 pixels, reverse the selection, and hit delete. This should create a fade effect around the edges of the smaller file which will work great to morph smoothly between the sounds. <BR> <BR>On an ending note, 8 bits is not the limit of your sound painting abilities in photoshop. Experiment, have fun. And the next time somebody tries to tell you that electronic music isn't art, you've got a good way to prove them wrong. <BR> <BR> <BR> -- Eric Hamilton &#40;aka: Dilvish&#41; <BR> <A HREF="http://www.mp3.com/erichamilton" TARGET="_top">http://www.mp3.com/erichamilton</A>

User avatar
Mike Zee
Posts: 766
Joined: Tue Mar 20, 2001 5:58 am
Location: NY, USA
Contact:

using Photoshop to process sound

Post by Mike Zee » Tue Jun 25, 2002 7:38 pm

howdy, man, <BR>sorry, have to say that I am just tooo lazzzzy to read the whole long and a bit borring post <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;">, just as a note about 'does HE fool the world' .... <BR>well, he does not fool the world on the subject that image/bitmap data editing application can process &#40;say: re-write&#41; any othere 'raw' digital data, and I mean any &#40;not just sound-file data&#41;. <BR>So technically speaking there is nothing to it really. <BR>Also the attraction of the idea to manipulate sound-files in graphical program is not new. Actually there are couple of specific applications to do that. I don't have it under my hand, but I remember some talks about it somewhere back few years ago or so... <BR>So, yes you can mess with your data in many ways. Is this a "creative way to find 'new sound textures'??? I don't see it that way. It's just random chaos, similar to , let's say: drop into the can some tooth-paste, some paint, some honey, some candy-wraps, some crashstone...shake it a bit, then through some of that on paper.... and you've got a creative painting. <BR>Some guys get hi doing this stuff...he he he, you know, just seating with computer and messing around with audio-files, then put it together into 'track'...., then call it 'ambient soundscapes' , these guys also often love to trash 'pop-music' or actually any 'kind of "organized" music, or "structured" music..., what ever we may call it. <BR> <BR>ok, <BR> <BR>later, <BR> <BR>respects, <BR> <BR>/Mike Zee <BR><A HREF="http://www.audiogalaxy.com/bands/mikezee/dub_la.htm" TARGET="_top">zee dub lab</A>

User avatar
interruptor
Posts: 1151
Joined: Sun Sep 04, 2005 11:46 pm
Location: Switzerland
Contact:

using Photoshop to process sound

Post by interruptor » Tue Jun 25, 2002 11:33 pm

i see you are rather skeptical about this one mike.. anyway if i find some time one day i will try this out. after all, something new and unheard of may come out. <BR>remember: the guys who invented the echo machine were trying to emulate natural room ambiance and they surely didn't have in mind what we are doing with these machines nowadays.. also the saxophone was created with the intention to have an instrument that can emulate the sweet sounds of strings for army brass bands and not for playing bebop, ska and other funky stuff. <BR> <BR>it's more likely than not of course that you are right in the end and nothing really useful comes out of it, but i WILL try this.. <BR> <BR>peace, love & unity <BR>daniel aka the interruptor

munky lee
Posts: 48
Joined: Sun Jul 22, 2001 11:10 pm

using Photoshop to process sound

Post by munky lee » Thu Jun 27, 2002 12:39 pm

sure I will try this also! <BR>no matter what the result is, I want to try. <BR> <BR>"making sounds with photoshop"... it rocks! <BR> <BR>munky lee

User avatar
KoCha
Posts: 259
Joined: Fri Jan 25, 2002 10:38 am
Location: Switzerland

using Photoshop to process sound

Post by KoCha » Thu Jun 27, 2002 2:40 pm

What ??? Make sounds with images ?? <BR> <BR>That can surely do strange sound!! <BR>What is the color difference between deep bass and highlty drums ? <BR> <BR>i'm going try this, when i found a photoshop... <BR> <BR>Jah Bless, <BR> <BR><A HREF="http://www.kocha.fr.st" TARGET="_top">Kocha</A>
KoCha, reggae-dub producer

Almighty Dub Records - Independent Reggae Dub Production
Open Dub Foundation - The WorldWide Dub Meeting

User avatar
Mike Zee
Posts: 766
Joined: Tue Mar 20, 2001 5:58 am
Location: NY, USA
Contact:

using Photoshop to process sound

Post by Mike Zee » Thu Jun 27, 2002 5:19 pm

heh heh, Daniel, man, i'm not really skeptical... i mean, 'skeptical' is not the word here <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR>I actually did try doing this type of things. I used to do vary crazy stuff, like saving audio-files "as..." some other types and then processing the data in other applicating, bring it back to sound.... The result is pretty much similar to something what you get out of random messing around dsp-effects, if you just take any sound file and start applying the whole bunch of effects with no order, and then just see what happens. You know there are 'electronic musicians' who just 'make their music' this way: the method is: "PUSH SOME BUTTON - and see what happens. Then do it again and and again, and you've got a "creative piece of art" ...<IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR>Well, man, sure you may get some strange and maybe cool sounds. Will it make you "music" any better or more creative... well, I'd say very little chance, but why not to try.... <BR>You know how it goes. <BR>I'll give you an example, ... <BR>I was working on a reggae track, not really dub, but just a groove. And I wanted some "creative" snare sound. So I've got the idea to try something different, then just searching through samples or applying mixing techniques. <BR>So I've got my little JVC-shelf system out. It has aux-in. So I've sent direct out of "snare track" chnl from my mixing desk to JVC, then I've placed huge metal air-fan &#40;size of a kik-drum or so&#41; against the wall. This fan has huge metal ring as a base of body, which can create some cool resonance, when the sound get inside... <BR>well, next I've placed a large metal coffe can right in the middle of that fan-ring , i've placed JVC-shelf spekers very close to that coffe-can, almost touching it, so the can seats in-between the speakers. I've placed the mic inside the coffe can, and the other mic close to air-fan's ring.... and well, start the drum loop-play. <BR>as result, I've got some vary and interesting snare sounds. <BR>Now....after I've made the track, and was listening to it later, ...I've got that thought: "And so what???? the snare is different, was allots of work done to get it, it's kinda cool, but really, the tune would be pretty much the same if it had just some "regula snare sample".... <BR>another words, it pretty much depends on music itself, and not that much on 'creative ways of creating sounds'. <BR>But , wait!!!! here's why I am telling all this in regard to 'photoshop sounddesign' <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR>You see, in my case, at least I had some 'common sense' of what I was doing, generally speaking, I was using acoustics to manipulate the sound. But using application like photoshop to manipulate audio-data would be the same if I, let's say, did something like this: <BR>"Place The Coffe-Can on gas-stove, fill it with water, turn the heat up, dip the mic into water, place spekers around, then turn heat up and down and see how it effects the sound". Boiling may actually create some cool effect... heh heh <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR> <BR>&#42;&#42;&#42;&#42; <BR>ok, guys, drope a note if you get something interesting to hear. <BR> <BR>later, <BR> <BR>respects, <BR> <BR>/Mike Zee

kozmik charlie
Posts: 1
Joined: Thu Dec 12, 2002 4:20 pm

using Photoshop to process sound

Post by kozmik charlie » Thu Dec 12, 2002 4:20 pm

A mic?! In water?!?! You can't be serious...

User avatar
Mike Zee
Posts: 766
Joined: Tue Mar 20, 2001 5:58 am
Location: NY, USA
Contact:

using Photoshop to process sound

Post by Mike Zee » Thu Dec 12, 2002 8:16 pm

kozmik charlie , heh heh, yeah, this is a strange topic here. Alchemists on the loose! <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR> <BR>&#42;&#42;&#42;&#42;&#42; <BR>lol <BR>&#42;&#42;&#42;&#42;&#42;, BUT, <BR>you actually can get a 'hydrophone', somethin' like this: <A HREF="http://www.bhphotovideo.com/product/224 ... C/REG/1936" TARGET="_blank">for under 200bucks</A> and start boiling ... <BR>btw, here's somthin' to read to kill'a'min'of'time: <A HREF="http://ccrma-www.stanford.edu/~blackrse/h2o.html" TARGET="_blank">Research in Underwater Sound With Focus On Musical Applications and Computer Synthesis</A> <BR> <BR>/respects, <BR>/Mike Zee

Sata Weeva
Posts: 29
Joined: Fri Oct 18, 2002 2:38 pm

using Photoshop to process sound

Post by Sata Weeva » Fri Dec 13, 2002 12:23 pm

Great news for summer. If it's too hot in the studio take the band and equipment to the local pool. <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;">

bop
Posts: 1
Joined: Mon Jan 20, 2003 7:45 am

using Photoshop to process sound

Post by bop » Mon Jan 20, 2003 7:45 am


lepetitmartien
Posts: 10
Joined: Mon Apr 07, 2003 12:08 am

using Photoshop to process sound

Post by lepetitmartien » Sun Apr 13, 2003 4:43 pm

You can use U&Isoftware Metasynth <BR> <BR>it's "simpler" ;&#41;

dUBsTA aka zAd
Posts: 4
Joined: Thu Jul 10, 2003 3:02 pm

using Photoshop to process sound

Post by dUBsTA aka zAd » Thu Jul 10, 2003 3:59 pm

hummmm .... photoshop for sound editing... <BR> <BR>I see your point,Eric, but as a "push-some-button-and-see-what-happens " musician <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> &#40;and bass player&#41; I'd say: first perfect your "usual" studio recording skills and then start messing around with weird gear and techniques ... and when in doubt,listen to Tubby's music , you'll find your answers <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <IMG SRC="http://www.interruptor.ch/cgi-bin/discu ... /happy.gif" ALT=":&#41;"> <BR> <BR>bless

eric hamilton
Posts: 1
Joined: Sun Sep 14, 2003 6:43 am

using Photoshop to process sound

Post by eric hamilton » Sun Sep 14, 2003 6:43 am

Haha, <BR> <BR>I wrote this article for Static Line. It's dedicated to electronic artists in the PC Demoscene. These guys are into pushing the limits of art and technology. <BR> <BR>Glad to see the article made a buzz somewhere unrelated to the magazine. Very cool. <BR> <BR>As for making weird sounds, these days I use reaktor and synthedit more than photoshop, but I still stand by the photoshop technique when you're just in the mood to experiement wildly with sounds, rather than concentrate on the core elements of the "song". <BR> <BR>If you're curious about my music, give it a listen.. You'll find some of it at <A HREF="http://www.dilvie.com/" TARGET="_top">http://www.dilvie.com/</A> <BR> <BR>Peace, <BR> <BR>dilvie &#40;aka, Eric Hamilton&#41;

slothbecker
Posts: 3
Joined: Sun Sep 14, 2003 7:34 pm

using Photoshop to process sound

Post by slothbecker » Thu Sep 18, 2003 2:47 pm

interest talk about what i've known as "data bending": fucking with sound via non-sound programs, or creating sound out of non-sound files. <BR> <BR>as for the idea that this is some sort of elaborate alchemy that can be done more simply with a plug-in: didn't scratch perry make a career out of this kind of alchemy? you'll often find that you can create brilliant effects as happy accidents that you couldn't have made if you had simply &#42;tried&#42; to create the effects using any kind of deductive reasoning. sometimes, getting stupid and doing things like mic-ing your fridge can be just a useful way to break "musicianly" habits. <BR> <BR>data bending isn't a science, though; you usually aren't going to be able to predict how X approach is going to effect Y instrument, so yr more likely to destroy a good bassline than improve on it by bitmapping and photoshopping it. but you can make some wicked unique sounds for drop-ins on yr dubs.

mynameismud
Posts: 1
Joined: Tue Oct 28, 2003 5:28 pm

using Photoshop to process sound

Post by mynameismud » Tue Oct 28, 2003 5:28 pm

the sound to image manipulation technique is prevalent in electronic music, mostly IDM. i dunno how itd be applied to dub music...but sounds interesting.

fattestbass
Posts: 13
Joined: Fri Mar 05, 2004 1:58 pm

using Photoshop to process sound

Post by fattestbass » Tue Mar 16, 2004 3:48 pm

I've used a program called coagulite to convert pictures to sound, it's a bit basic though... <BR>One of afex twins tunes shows a picture of him if you do a time &#40;X&#41; amplitude&#40;Y&#41; frequency &#40;Z&#41; plot &#40;FFT in cool edit&#41;of the sound- I'll try and find out which one it is.... <BR>On the under water mic - just use any old cheap mic and bung a couple of condoms around it &#40;NOTE CHEAP MIC.....i'm not taking the blame if it all goes wrong..&#41;. <BR>hope these things may amuse some of you, so enjoy <BR>Pat

Simplifier
Posts: 5
Joined: Wed Apr 27, 2005 3:26 pm

using Photoshop to process sound

Post by Simplifier » Tue Dec 27, 2005 11:05 pm

I'd be very tempted to use cheap condoms, too.

Post Reply