using Photoshop to process sound
Posted: Tue Jun 25, 2002 4:47 pm
I am not shure yet if this guy is trying to fool the world - but his idea is definitely tempting! Let's use the image processing tool photoshop to manipulate wav-files: <BR> <BR>--=--=------=--=------=--=---- <BR> Painting Sound <BR> By: Eric Hamilton <BR>----=--=------=--=------=--=-- <BR> <BR>Some people go to great lengths in the search for new sounds. As a teenager I was inspired by tales of Jimi Hendrix constantly looking for new ways to mangle his guitar sounds, and the accounts of electronic bands like New Order piping everything you can concieve through their Moog's filters. One day I was reading an issue of Mix Magazine when I saw an ad for a new type of synthesis program. It was a program that actually allows you to paint sound. I got to thinking, and I figured out that Photoshop has a raw file import feature, and Soundforge has a raw file output feature. I put two and two together and started to experiment. Here's what I figured out about painting sound. <BR>The hardest part is in the conversion. There's no way to achieve predictable results unless you can export an audio file, import it into photoshop, export it again, and import it into soundforge without changing the sound. There are actually several ways to accomplish this feat, but I will make it as easy as possible. <BR> <BR>Start by loading a sound into soundforge. Select File->Save as in the drop-down menu. You'll be presented with the save as dialog box. Change "Save as type" to "Raw File (*.raw, *.*)". Give your file an apropriate name and save it in a directory where you'll be able to find it again easily. I created a sub-directory in my sample creations directory called, "photoshop", but as long as you can find it, anything will do. Next, you'll be presented with a dialog asking how to save the file. Select 8 bit, unsigned, mono, and click "Okay". <BR> <BR>The next step is to load the file into photoshop. Choose File->Open from Photoshop's drop-down menu, and you'll be presented with the Open dialog box. Browse to the directory where you saved the raw audio file, and make sure the "Files of type" select box is set to "All Formats" or "Raw (*.RAW)". Select the file, and click "Open". Photoshop will pull up the "Raw Options" dialog to collect some information about your file's dimensions, channels, and header info. Normally, it will make a guess at the dimensions automatically. The dimensions that photoshop guesses at have always worked well for me, but once in a while the math doesn't pan out. If this is the case, photoshop will present you with blanks. Rather than trying to work out the math (which will force you to cut out part of the sound), you'll want to insert silence in soundforge to create a good starting point for photoshop. You may have to experiment a bit to get this right, but here's a tip: It's easier for me to work backwards. When photoshop gets stumped, create a new file (greyscale), and take a guess at how big it will need to be to handle your sound. 500x500 in greyscale is aproximately 5 1/2 seconds of audio. Save the file, import it into soundforge, zoom out so you can see the entire file, press the End key, and check the duration. If it's not long enough, try creating a larger file. If it's too long, that's fine. It'll work. Insert silence into your sound file until both files are the same size, re-save the raw audio file, and import into photoshop. Make sure it's set to 8 bits and no header. Click "Okay", and you should see an interesting greyscale image with some cool patterns. The top of the image will likely contain some blacks and whites, with a gradual fade as the image gets more and more grey towards the bottom. Saving the file from photoshop isn't such a hastle. It already knows what it needs to encode the raw file. I use <BR> the save-as dialog and choose a new name for the modified file so that you don't have to do the conversion all over again if you don't like the <BR> results. Once it's saved, load the file back into sound forge and compare with the original. Use the same settings for your soundforge file open dialog (it'll bring up the "Raw Options" again), and you should be set. <BR> <BR>If both files sound the same, you're on the right track. Now let's take a look at what you can do to a sound in photoshop now that you know how to load and save. The sound maps to the image in the following ways (from what I've been able to discern): Each greyscale pixel represents one sample. If the sample is dark, it will be low in amplitude. If it is bright, the amplitude will be high. Thus, the contrast of the image represents the over-all amplitude of the sound. Another interesting note is that Photoshop does not interpret negative amplitude as a negative color value. Instead, everything is positive, making grey the zero point in terms of audio. 128, 128, 128, to be exact in the RGB Value. <BR> <BR>The sound is mapped to the image in order from beginning to the end left-to-right and top-to-bottom. The beginning of the sound generally has the attack characteristics, and thus the most contrast in photoshop (for percusive or plucked sounds). Here you may find you can do things with your sound's attack and decay properties that you never thought possible. For example, you can create a new channel and apply gradients running from top to bottom that will change the dynamics over time. You can twist and bend sounds in photoshop in ways unlike any other sound editor on the planet. To apply a gradient that will alter the sound's attack, open the channels palette, create a new channel, and name it "Selection mask". Then Set your foreground color to white, and your background color to black, apply a gradient from top to bottom, and <BR> re-select the original channel. Click Select->Load Selection, and choose "Selection Mask" from the drop-down menu. Click "Okay". Make sure your background color is set to 128, 128, 128, and hit delete. And you thought ADSR envelopes were powerful. <BR> <BR>The dimensions of the image have a large impact on the way that the audio will sound. Genarally a combination of height and width determines the length of the sound (naturally, since more sample data means a longer duration). Something you might not guess right away though is that altering the width of the image *can* modify the fundamental frequency. The precise mapping for this is actually variable, though, depending upon the pattern of light and dark sounds in the image. I suggest you experiment, and use soundforge or another audio tool to tune the resulting sounds. Simply applying a filter can change the fundamental percieved frequency of your sound drastically. <BR>My main point here is that just as in real life, size matters when you're painting sounds. Altering your images dimensions will also alter the formants, the same way that playing a picolo produces different frequencies than playing a bassoon. <BR> <BR>There is a lot to be said for experimentation, but I must warn the beginner, photoshop is a powerful program that is not designed specifically to make audio files sound good. It does have a lot of <BR> features that have some very cool and interesting effects on your sound files, but be warned, it is very easy to create noise in photoshop that will not sound pleasant at all. <BR> <BR>For example, in many photographs and images, there is a high degree of contrast, and many pixels placed close to each other that are not similar in value. I'll call this texture from now on. In audio, there is generally much less texture. Texture's number one enemy in photoshop (and your number one friend in fighting it) is the gaussian blur located in the Filter->blur menu. Another way to fight texture is to reduce the contrast of the image, but this method is less effective in that it reduces texture by reducing the amplitude of your sound, which is not necesarily the result you're after. The gaussian blur does have some <BR> side-effects you may not be prepared for, though. In addition to smoothing out the rough edges and removing distortion, it will also alter the formant frequencies of your sound. This can be good or bad, and can be fixed or enhanced in soundforge to some extent with the graphic EQ feature - if you're good at identifying which frequency to boost or cut (something I'd start practicing right now if you're not). <BR> <BR>For the most part, sounds you import will have blacks and whites up top, with a smooth fade to grey as it goes down. Images you might load will probably look quite different. This basic sound shape is a result of how sound works in nature: attack, sustain, and decay. That's why most synthesizers come with envelopes that let you edit Attack, Decay, Sustain, and Release values (ADSR Envelopes). It's a good rule of thumb that unless you're planning to create a looping wave and apply ADSR envelopes in you sampler or soundforge, you should try to mimic the behavior in photoshop. <BR> <BR>Without altering the general shape of your sound, there are many things you can do that will wildly change or enhance the frequency spectrum. One of the most dramatic examples is the color curve feature in photoshop (Image->Adjust->Curves). Also try Image->Adjust->Auto Levels to try to maintain a good center and maximize your signal. Each time you run the auto-levels command, you may need to re-adjust the center manually using the brightness/contrast dialog. To do this, I like to zoom in on the lower right corner of the image, use the dropper tool to select the last pixel, and then double click the forground color on the toolbar to view the color settings. If the RGB values are anything but 128, go into the brightness/contrast dialog and add or subtract to compensate. You may also want to adjust the contrast to slightly reduce the amplitude and avoid clipping effects. <BR> <BR>Some effects that have produced cool sounds are the Filter->Distort->Twirl effect, Filter->Render->Clouds or Filter->Render->Difference Clouds. You can achieve a strange blocky sound by running any plugin that has the effect of reducing the number of distinct colors in the image (such as Cutout, Stained Glass, or Crystallize). <BR> <BR>Another thing you might want to try is the photoshop twist on sound morphing. To begin, choose two sounds that are roughly the same frequency. Load them both into Soundforge and add silence into the shorter sound until the two sounds share identical length. From there, import them into Photoshop with identical parameters. There are multiple ways you can morph the sounds. My favorite is to copy the shorter sound into a new layer on the larger sound.. position the smaller image so that it comes in where you want it to. Adjust the new layer's opacity to let some of the other sound mix into it. Now set up a selection on the smaller sound, and set the selection feather value (in the Selection menu) to something like 30 or 50 pixels, reverse the selection, and hit delete. This should create a fade effect around the edges of the smaller file which will work great to morph smoothly between the sounds. <BR> <BR>On an ending note, 8 bits is not the limit of your sound painting abilities in photoshop. Experiment, have fun. And the next time somebody tries to tell you that electronic music isn't art, you've got a good way to prove them wrong. <BR> <BR> <BR> -- Eric Hamilton (aka: Dilvish) <BR> <A HREF="http://www.mp3.com/erichamilton" TARGET="_top">http://www.mp3.com/erichamilton</A>