I finished and presented my animation, 'Drawing Music In Time' in Visual Music and Project Development Studio classes.
Presentation file for Project Development Studio class.
DrawingMusic - Designing characters and sounds
[April 9, 2009]
I’m working on designing characters and their sounds. I have two ideas of connecting visual and sound. First, I’m considering the color theory of Kandinsky. Each instrument or voice has its own characteristic timbre. In his 1912 book, on the Spiritual in Art, Kandinsky considered how these might relate to colors. Second, I’m thinking of making sound related to all kinds of ‘water’. I’m still not sure which sounds will work better.
These are images of characters.
DrawingMusic - Draft Storyboard
[April 6, 2008]
I made first draft storyboard for final project which will be a kind of visual music. I got inspired by Paul Klee's work. Here is my research on Klee’s work for Visual Music class paper assignment.
http://itp.nyu.edu/~sjh376/VisualMusic/Klee.pdf
I need more researches on sound and advice how to make it.
And, last Wednesday, I finished the basic programming for painterly expression which I hope to apply to my animation.
Painterly Expression
I tried to create painterly expression, especially oil painting, for my second midterm project of Pixel by Pixel class.
There are various painting styles. Then, what makes the difference between oil painting and other paintings or drawings?
First, I think, the materials of oil paints and oil painting brush make very unique features of oil painting. The thick oil paints produce even 3D effect on canvas and the brush creates brush texture due to its stiff hair. To get closer to the features, I took several pictures of real oil painting brushes and changed their colors to gray tone which would be able to accept the colors of source image.
Second, in the process of real painting, painter uses different size of brushes, usually painting broad area first with big size of brush and then getting into details with smaller one even though it is different depending on painters’ style. Therefore I made three layers of three brush sizes, big, middle and small. After applying first big brush, I compared the brightness of the screen image to the one of source image. If the difference of the brightness is big, I think, that means ‘details’ which need more painting with smaller brush. After applying second middle brush, I did the same way before third small brush.
<Image painted by first big brush>
The third thing what I did was about ‘color’. I think painter can’t replicate the color of reality on canvas because he/she has many limitation to do that; the limitation of perception, materials he/she uses and his/her hand skill. Instead those limitations make painter’s own style. Thus the colors we can see on paintings are very limited and I tried reducing the numbers of colors for brush strokes. By the way, reducing colors without considering and doing research on colors of real painting resulted disappointing images. I thought manipulating color had lots of possibility of expressing more stylish painting so I decided not to use this idea for this time and put aside for further work.
The last one is about ‘outline’. Outline may not be necessary in some paintings but some may need it. To draw an outline, I used edge detection and tried to draw different brush with other brushes on that edge. However, I continuously got an error. I think I can use edge detection both to draw outline with darker color than neighbor colors and to make the direction of strokes follow the edge’s(now the direction of stroke is random). That will be also the next step of my challenge in painterly expression project.
Here is the code.
http://itp.nyu.edu/~sjh376/testApp.cpp
The analytic approach of this project is based on the research experience on NPR(Non-Photorealistic Rendering) technique, when I was a graduate student of ChungAng Unuversity in Korea. The research was a joint project between the lab which I was belonged to and was mainly consisted of artists and designers and the lab consisted of engineers. I took the responsibility of analyzing art, especially oil painting in technique and Impressionism in genre, and converting the facts from the analysis to the guidelines and materials which were applicable to the process of developing algorithm. The research resulted ‘Painterly Rendering with Designed Imperfection’, the installation artwork exhibited in Guerilla Studio, SIGGRAPH 2006. Also my master’s thesis is ‘Study for Non Photorealistic Rendering based on Aesthetic Analysis’.
The motive which had made me get interested in NPR technique was the idea of painterly abstract animation. I’ve wanted to apply NPR to my animation. Therefore, doing midterm assignment, I was excited to realizing some parts of my analysis with openframeworks by myself. Applying current code to moving image is another matter because oil painting and the animation using oil paints have different process of making and thus give us different feeling. That’ll be also my next challenge until I can realize what I imagine.
DataFlow MidtTerm Project - Clapping Music
[March 13, 2008]
I made a visual music inspired by clapping music of Steve Reigh for my DataFlow midterm project.
n the clapping music, there are two rhythms playing together, one basic rhythm and the other based on the basic one. First, the two rhythms repeat 3 times with same beats and then the second one shifts forward by one beat, that is every count of ’36’ makes different combination of rhythm.
Sounds are samples of 4 percussion sounds and opened according to mouse X and Y position. So, 4 different mixture of percussion can be created according to user’s mouse control.
In visual part, I tried it in a similar way. First, I made 24 rectangles in Gem window. Every rectangle represent every beat and rest. At every beat, the color of rectangle change from gray to pink and then return to the original color of gray. The sequencer for visual is little bit different with that of music because the rest also should send data to the rectangle to return the color.
DataFlow MidtTerm Starts - Clapping Music
[February 18,2009]
I’m planning to experiment combining music with animation. I’ve got inspired by ‘Clapping Music of Steve Reigh’.
http://www.youtube.com/watch?v=BhhIZscEE_g
This music is composed of two rhythms and changed according to the shift of second rhythm.
I’m thinking applying musical elements to graphical and animation elements. And in order to do this, I’m doing research on OSC(open sound control).
Hi, Mr. Tech!
[December 12, 2008]
My long time experience in traditional and digital animation drove me into the idea that user changes several key factors of an animation such as keyframes, characters or environment and gets involved in highly interactive experience rather than just watches what happened in a screen. I thought combination of physical character(kind of a puppet) and 'Processing(an open source programming language)' together would work great to make the idea realized.
With this concept, I could meet the partner, Adi who also had interest in animation. We did brainstorming and lots of research on a wide range of works related to animation and interaction.
However, I, first, got a little doubt how to make a physical character alive in a way of interacting between user input and movement of the physical character. Several ideas hit me on the head and I picked up one, spinning LED matrix which creates kinds of hologram. That means user draws a shape on a input device(ex. using LEDs as a photo diode) and an animated illusion shows up on the hologram based on the user’s drawing.
At this point of time, Adi suggested a wonderful idea, a straw doll, controlling the arms and legs of a character by pulling down strings connected inside the doll. It gave me a clue to solve my doubt and got back my eagerness to give a physical character a life on the table.
After that, we started to look for any possibility to realize these two options, LED hologram and a straw puppet. In the processes of testing, we found that the persistence of vision was much strong than we had expected. I think this can be solved if we use a device limiting the intensity of LED light or control the blinking speed of LEDs. But we had to put this option away because it required much time and technical knowledge to solve.
On the contrary, the doll showed us much more possibility to be realized in this given period. With lots of trials and errors, we reached the conclusion that we moved to another design of physical character because the straw doll was hard to be controlled with kinds of sensors.
Inspired by a design of commercial doll, cha-cha, I got an idea using metal springs and simple plastic body.
Adi had a amazing skill to make a physical thing. We could make a really cool doll, Mr. Tech, using metal springs, acrylic plates and wood. We put a dot matrix expressing happy and sad face reacting to some conditions of virtual reality in Mr. Tech.
We built Mr. Tech got a life in virtual reality by using 4 flex sensors and that LED dot matrix. All of these tests with Mr. Tech were quite successful and so we began to create exciting story in which Mr. Tech lived.
Testing with Mr. Tech was really fun but wiring him to computer gave user heavy constraints to joy of interaction. So I tried to use Xbee for wireless serial communication.
I was eager to implement wireless networking but time didn’t allow me to give a big present to Mr. Tech. I’m so sorry to him because he lost the chance to move and interact more freely with users.
We finally introduced Mr. Tech at my class, Physical computing and ICM, on early December, 2008. It was a great work and exciting experience. Thanks to my class mates, Kate and Shawn, and especially Adi.
Here is the link for presentation.
http://itp.nyu.edu/~sjh376/Final/Last/FinalPT.pdf
Here is the link for the test movie which I made by controlling Mr. Tech by myself.
http://itp.nyu.edu/~sjh376/Final/MTMovie.mov
Final Project - Test of Serial Communication
[November 25, 2008]
We are thinking the reaction of Mr. Tech to some events in processing for better interactive animation. So, as a test, we made the simple codes to send serial data from processing to arduino at some point and turn on a LED at that time. We could get the right result which encourages us to develop the project with this idea.
Final Project - Mr. Tech with animation
[November 21, 2008]
We, Adi and I, tested several sensors for Mr. Tech’s movements and animations in processing. And we made the storyboard consisted of 4 scenes. Bicycling, Drawing, Wether and Drumming. Now, we think user can get 4 input to control both the physical character and animation at the same time; two arms and two legs. In every scene, there are several events which user can control and make their own animation by manipulating the 4 inputs.
LED Matrix Test
[November 17, 2008]
I tested the feasibility of LEDs, I made 8 X 11 led dot matrix controlled through dip switches. As a simple test, I used dip switch instead of micro controller. I thought I could control each LED with the corresponding switch. But it didn’t work. I need more switch to control all LEDs. Anyway, the matter of persistence of vision is still the problem. I could not keep developing this LED project as to what I had planned.
Final Project Starts
[November 17, 2008]
Adi and I are working together for the Final Project of both ICM and Pcomp. Our common interests are ‘animation’ and ‘interaction’. We did research on traditional animation and contemporary art work related to animation. For now, we have two options to develop. The spinning LED dot matrix and Mr. Tech.
First, we tested the feasibility of LED matrix which would display images drawn on input device by user. In addition to the LED display, the same drawings will appear in processing sketch and start irrespectively animating. User drawing creates both illusion on LEDs and animating objects on processing sketch simultaneously. However, we have to overcome one technical problem in order to realize this idea. At the LED matrix test, we met unexpected result, excessive persistence of vision while lights were spinning. One solution we will try is that whole image will be inverted and turning LEDs off will make a shape.
Second, we tried to make a doll, Mr. Tech, controlled by user. However, we couldn’t make the exact plan for what kind interaction and narrative we will show yet.
This is the presentation file.
((H))Art - Midterm Project
[October 24, 2008]
"Your heart beating makes the world full of life.
See, hear and enjoy your own beat."
My group, Jason, Winslow and I, presented our project “((H))ART” in physical computing class yesterday. With the help of a heart rate monitor, “((H))ART” is both a work of biofeedback and interactive art.
Flex Sensor - Tone Generation
I made octave generator playing an octave by controlling flex sensor for the weekly assignment of Physical computing class.
Lab : Electronics
1. Measuring Voltage
Wiring a 7805 5-volt voltage regulator on a breadboard and connect it to power
2. A Basic LED Circuit
Adding an LED, a switch, and a resistor in series
3. Components in Series
Connecting two LEDs in series from power to ground
4. Components in parallel; measuring amperage
Connecting three LEDs in parallel
[September 25, 2009]
About me
I’m a graduate student of ITP(the Interactive Telecommunications Program) of New York University. Through this program, I continue to do research on kind of interactive virtual reality. I’ve dreamed that there is the world where people can make their own stories with their own narratives derived from their very own interactions with my artwork. The world, the story or the narrative in which people walk, breathe and live in their own ways is the place I’m eager to reach.
To make steps for this, I have mainly focused on 4 subjects:
- programming skill - the necessity to build the virtual reality and make it interactive with people
- tangible interface - the bridge between reality and virtual reality
- non photo realistic rendering - for visual, one of the 2 elements consisting of the world( visual and auditory )
- connection between sound and visual
Through researches on these areas, I completed several projects.
The projects at ITP include ‘Mr. Tech’, interactive animation controlled by physical character, ‘Painterly expression’ in openFrameworks, data visualizations in processing, visual music project using Pure Data.
I also participated in the project, ‘Painterly Rendering With Designed Imperfection’, which exhibited in SIGGRAPH 2006.
Animation
I can never forget the time when I first saw the two short animations, ‘PARADISE’ made by Ishu Patel and ‘THE STREET’ by Caroline Leaf, in the Moving Image Design class of SNU. It has changed my whole life. Since then, I’ve enjoyed experimenting on various kinds of animation techniques.
- traditional techniques(cutout, clay, paint-on-glass, sand, puppet)
- digital techniques(2D, 3D, web based and interactive animation)
My works in the course of the experimentation:
‘When I met them’(2000, film, 0:04:00, Oil pastel),
‘circle, triangle, square’(2001, film, 0:03:30, Pen drawing),
‘Flower dance’(2006, DV 6mm, Color, 0:02:43, 3D Computer),
‘HArt’(2008, Interactive animation with heart beat sensor),
‘Mr. Tech’(2009, Interactive and real-time),
‘Drawing music in time’(2009, Color, 0:04:30, 2D Computer).
Cinema
I studied film theory, film making, digital sound making and traditional storytelling in non degree film institute. I have experience of making an independent short film as an assistant director. Based on my previous study on traditional cinema, I’m still doing research for future cinema.
'FUTURE CINEMA', 'EXPANDED CINEMA'
Interactive storytelling
These are my questions:
- How I can make my animation more interactive with viewers?
- Can my animation allow viewers to create their own stories irrespective of my story?
NPR
‘Painterly Rendering With Designed Imperfection’ in the Guerilla studio of Siggraph 2006. Movie file
'Painterly expression' in ITP, 2009
C++
The interest in interactivity and NPR pushed me to study programming. It was not easy but became a necessity for me to go further. And, now, I enjoy it and think it’s so fascinating.
Physical computing
The Physical Computing class in ITP showed me another world which can broaden my creativity.
Major
- Bachelor of Visual communication design in Seoul National University(SNU), Korea
- Master of Image engineering in Chung Ang University, Korea
- Currently pursuing Master of Professional Study in ITP(the Interactive Telecommunications Program), Tisch school in NYU
Subscribe to:
Posts (Atom)